Jul 06 15:24:37.042044 ip-10-0-141-57 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 15:24:37.042244 ip-10-0-141-57 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 15:24:37.042340 ip-10-0-141-57 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 15:24:37.043023 ip-10-0-141-57 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 15:24:47.121913 ip-10-0-141-57 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 15:24:47.121935 ip-10-0-141-57 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8c3eb94f39d846a69a2a8ac7da6af58e --
Jul 06 15:26:46.304998 ip-10-0-141-57 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.473465    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478033    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478052    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478056    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478059    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478063    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478066    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478071    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 15:26:48.529964 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478073    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478076    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478079    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478082    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478085    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478089    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478092    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478094    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478097    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478100    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478103    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478107    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478112    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478116    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478120    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478123    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478126    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478129    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478132    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478134    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 15:26:48.531641 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478137    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478139    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478142    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478144    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478147    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478150    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478154    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478157    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478160    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478162    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478167    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478171    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478175    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478178    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478180    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478183    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478185    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478188    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478190    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 15:26:48.532723 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478192    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478195    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478199    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478202    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478205    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478207    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478210    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478213    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478215    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478218    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478221    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478223    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478226    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478228    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478231    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478233    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478236    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478238    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478241    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478243    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 15:26:48.533950 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478245    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478248    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478250    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478254    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478256    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478259    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478264    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478267    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478269    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478272    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478274    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478277    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478279    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478281    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478284    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478286    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478289    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478292    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478294    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478298    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 15:26:48.534999 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478300    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478303    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478305    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478308    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478311    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478313    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478316    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478318    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478320    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478323    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478325    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478328    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.478330    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479116    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479124    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479127    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479130    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479133    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479136    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479140    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 15:26:48.536000 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479143    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479145    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479148    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479150    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479153    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479155    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479158    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479161    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479164    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479167    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479169    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479171    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479175    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479178    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479180    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479183    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479185    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479188    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479190    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479193    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 15:26:48.538200 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479196    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479198    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479200    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479203    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479205    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479208    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479210    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479213    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479216    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479219    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479222    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479224    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479227    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479230    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479233    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479235    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479238    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479241    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479243    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 15:26:48.538844 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479246    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479249    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479253    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479257    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479260    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479262    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479265    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479267    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479270    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479272    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479279    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479281    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479284    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479287    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479289    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479292    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479296    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479300    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479303    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 15:26:48.644519 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479306    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 15:26:48.556481 ip-10-0-141-57 systemd[1]: Started Kubernetes Kubelet.
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479309    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479313    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479316    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479319    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479322    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479325    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479329    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479332    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479335    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479337    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479339    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479342    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479344    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479347    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479350    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479352    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479354    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479357    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479359    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 15:26:48.645539 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479362    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479368    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479370    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479374    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479376    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479380    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479382    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479385    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479387    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479390    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479392    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479395    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479398    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479401    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479827    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479833    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479836    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479840    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479844    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 15:26:48.646330 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479848    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479851    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479854    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479857    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479860    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479863    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479866    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479868    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479871    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479874    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479876    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479879    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479882    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479885    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479887    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479890    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479894    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479897    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479900    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 15:26:48.647174 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479902    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479905    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479907    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479910    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479913    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479915    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479918    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479921    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479923    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479925    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479928    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479930    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479933    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479935    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479938    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479941    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479944    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479946    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479949    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479951    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 15:26:48.647995 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479954    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479957    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479959    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479962    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479964    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479967    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479969    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479972    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479974    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479979    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479982    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479985    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479987    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479990    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479993    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479995    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.479998    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480001    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480004    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480007    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 15:26:48.648777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480009    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480012    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480014    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480017    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480019    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480022    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480025    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480030    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480032    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480035    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480038    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480041    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480045    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480048    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480050    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480053    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480056    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480060    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480062    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 15:26:48.649549 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480066    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480068    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480073    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480075    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480078    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480080    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480083    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480086    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480088    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480091    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480094    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480096    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480099    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480101    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480104    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480106    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480200    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480223    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480233    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480238    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480245    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 15:26:48.650325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480249    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480253    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480258    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480261    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480264    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480268    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480271    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480274    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480277    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480280    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480283    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480286    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480289    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480294    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480297    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480301    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480304    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480307    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480311    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480314    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480317    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480321    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480324    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480327    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 15:26:48.651183 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480330    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480333    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480336    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480340    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480343    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480345    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480349    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480353    2580 flags.go:64] FLAG: --enable-server="true"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480356    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480363    2580 flags.go:64] FLAG: --event-burst="100"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480366    2580 flags.go:64] FLAG: --event-qps="50"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480370    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480373    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480376    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480379    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480382    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480385    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480388    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480391    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480394    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480397    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480401    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480404    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480407    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480410    2580 flags.go:64] FLAG: --feature-gates=""
Jul 06 15:26:48.651996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480414    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480417    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480420    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480423    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480426    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480429    2580 flags.go:64] FLAG: --help="false"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480432    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-141-57.ec2.internal"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480435    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480438    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480441    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480444    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480447    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480450    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480453    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480458    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480461    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480463    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480467    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480470    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480473    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480476    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480479    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480482    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480485    2580 flags.go:64] FLAG: --lock-file=""
Jul 06 15:26:48.652795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480488    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480491    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480494    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480499    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480504    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480507    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480510    2580 flags.go:64] FLAG: --logging-format="text"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480513    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480516    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480519    2580 flags.go:64] FLAG: --manifest-url=""
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480522    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480527    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480530    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480534    2580 flags.go:64] FLAG: --max-pods="110"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480537    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480540    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480543    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480546    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480549    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480552    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480555    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480563    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480567    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480570    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 15:26:48.653592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480573    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480576    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480583    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480586    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480589    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480592    2580 flags.go:64] FLAG: --port="10250"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480595    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480610    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e2c94c1b892c82ec"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480613    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480616    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480618    2580 flags.go:64] FLAG: --register-node="true"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480622    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480625    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480629    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480632    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480635    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480639    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480642    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480645    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480648    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480650    2580 flags.go:64] FLAG: --runonce="false"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480653    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480656    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480659    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480663    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 15:26:48.654371 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480665    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480668    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480671    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480674    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480677    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480681    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480683    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480686    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480689    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480692    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480708    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480714    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480717    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480720    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480724    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480727    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480730    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480732    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480735    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480738    2580 flags.go:64] FLAG: --v="2"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480744    2580 flags.go:64] FLAG: --version="false"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480747    2580 flags.go:64] FLAG: --vmodule=""
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480751    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.480755    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480873    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 15:26:48.655209 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480876    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480879    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480882    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480885    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480888    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480890    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480893    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480895    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480898    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480900    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480903    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480905    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480909    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480912    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480914    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480917    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480919    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480922    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480926    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480929    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 15:26:48.656021 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480931    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480934    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480936    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480939    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480941    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480943    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480946    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480950    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480952    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480955    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480957    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480960    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480963    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480965    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480968    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480972    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480975    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480978    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480980    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480983    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 15:26:48.656833 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480986    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480988    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480991    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480993    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480997    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.480999    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481002    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481004    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481007    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481009    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481011    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481014    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481017    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481019    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481022    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481024    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481027    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481029    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481033    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 15:26:48.657625 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481038    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481041    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481043    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481046    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481049    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481051    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481054    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481056    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481059    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481062    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481064    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481068    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481071    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481074    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481077    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481080    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481082    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481086    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481088    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481091    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 15:26:48.658457 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481094    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481096    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481098    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481101    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481103    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481106    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481108    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481111    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481113    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481116    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481118    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481121    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481125    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481127    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481130    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481132    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481134    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481137    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 15:26:48.659282 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.481140    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.481816    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.491776    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.491799    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.491822    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.491828    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491902    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491908    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491911    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491914    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491917    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491926    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491929    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491932    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491935    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 15:26:48.660121 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491937    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491940    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491943    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491946    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491949    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491951    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491954    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491956    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491958    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491961    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491964    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491967    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491971    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491974    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491977    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491979    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491982    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491984    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491987    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491990    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491992    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 15:26:48.663684 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491995    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.491997    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492000    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492002    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492005    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492007    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492009    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492013    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492016    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492018    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492021    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492023    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492026    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492029    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492032    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492034    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492037    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492039    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492042    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492044    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 15:26:48.664517 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492047    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492050    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492052    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492057    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492060    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492063    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492066    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492068    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492071    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492074    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492076    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492078    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492082    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492086    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492089    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492092    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492094    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492097    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492099    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492104    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 15:26:48.666095 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492106    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492109    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492113    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492117    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492120    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492123    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492126    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492128    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492131    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492134    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492137    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492140    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492142    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492145    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492147    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492152    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492155    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492157    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 15:26:48.667351 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492161    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492165    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492168    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492171    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492173    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492176    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492178    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492181    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492184    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492186    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492189    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.492194    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492315    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492322    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492325    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 15:26:48.668555 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492327    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492330    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492332    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492336    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492338    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492341    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492343    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492346    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492348    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492351    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492353    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492356    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492358    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492361    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492363    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492367    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492369    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492372    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492374    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 15:26:48.669343 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492377    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492379    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492382    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492384    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492387    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492389    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492392    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492395    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492397    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492400    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492402    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492407    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492411    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492414    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492416    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492419    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492421    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492424    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492426    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492429    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 15:26:48.772777 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492432    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492434    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492436    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492440    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492442    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492445    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492447    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492449    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492456    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492458    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492461    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492463    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492466    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492468    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492471    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492473    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492475    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492478    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492480    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 15:26:48.773569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492483    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492485    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492488    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492491    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492494    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492497    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492500    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492502    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492505    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492509    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492511    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492514    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492517    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492519    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492521    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492524    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492526    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492530    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492532    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492534    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 15:26:48.775569 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492537    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492541    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492544    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492547    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492549    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492552    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492554    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492557    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492561    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492564    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492566    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492569    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492571    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492574    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492577    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492579    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492583    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 15:26:48.776403 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:48.492585    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.492590    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.492746    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.498244    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.499721    2580 server.go:1019] "Starting client certificate rotation"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.499848    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.499902    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.502388    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.506092    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.516357    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.521166    2580 log.go:25] "Validated CRI v1 image API"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.522017    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 15:26:48.777573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.526067    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 15:26:48.785471 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.529301    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cc9c286f-29c2-4511-b73d-90cd67fa03c5:/dev/nvme0n1p3 e2812d6e-679f-4c8c-85c5-9e03fc3cd601:/dev/nvme0n1p4]
Jul 06 15:26:48.785471 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.529332    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.536678    2580 manager.go:216] Machine: {Timestamp:2026-07-06 15:26:48.534864692 +0000 UTC m=+0.347727462 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3189605 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec275ed585e4d7e285ed3603b57952a6 SystemUUID:ec275ed5-85e4-d7e2-85ed-3603b57952a6 BootID:8c3eb94f-39d8-46a6-9a2a-8ac7da6af58e Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c6:8d:3b:dd:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c6:8d:3b:dd:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:10:51:43:9e:dc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.536840    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.536971    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.539455    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.539501    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-57.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.540247    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.540258    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.540266    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.540926    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.542352    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545426    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545555    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545586    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545596    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545610    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545619    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.545629    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 15:26:48.786693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.546050    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.546081    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.548170    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.550062    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.550094    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551336    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551349    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551356    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551362    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551371    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551376    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551385    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551392    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551398    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551404    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551420    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.551427    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.552225    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.552232    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.554193    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vhdlr"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.555500    2580 server.go:1268] "Started kubelet"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.555607    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.555678    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.555927    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.556990    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-57.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.557124    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.557262    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.557651    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.558325    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.560611    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-57.ec2.internal.18bfbcd6e1920492  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-57.ec2.internal,UID:ip-10-0-141-57.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-57.ec2.internal,},FirstTimestamp:2026-07-06 15:26:48.555472018 +0000 UTC m=+0.368334787,LastTimestamp:2026-07-06 15:26:48.555472018 +0000 UTC m=+0.368334787,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-57.ec2.internal,}"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.562495    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vhdlr"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.562673    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.566636    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.566681    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.566796    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.567312    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.567327    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.567311    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.567466    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.567476    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 15:26:48.788671 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568114    2580 factory.go:153] Registering CRI-O factory
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568125    2580 factory.go:223] Registration of the crio container factory successfully
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568173    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568180    2580 factory.go:55] Registering systemd factory
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568185    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568201    2580 factory.go:103] Registering Raw factory
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568210    2580 manager.go:1196] Started watching for new ooms in manager
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.568797    2580 manager.go:349] Starting recovery of all containers
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.569299    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.569340    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-57.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.578798    2580 manager.go:354] Recovery completed
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.583096    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585198    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585235    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585247    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasSufficientPID"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585763    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585774    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585791    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.585806    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.586568    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.586597    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.586616    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasSufficientPID"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.587029    2580 policy_none.go:49] "None policy: Start"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.587042    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.587053    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:48.588089    2580 policy_none.go:47] "Start"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.667197    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.767496    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:48.789868 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.776098    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-57.ec2.internal\" not found" node="ip-10-0-141-57.ec2.internal"
Jul 06 15:26:48.894632 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.868422    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.006982 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:48.968725    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.123157 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.069378    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.239492 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.169792    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.366362 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.270231    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.373820 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.370634    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.489592 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.471224    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.504216 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.499469    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 15:26:49.610163 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.568328    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 15:21:48 +0000 UTC" deadline="2027-12-30 19:46:05.660969268 +0000 UTC"
Jul 06 15:26:49.610163 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.568370    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13012h19m16.092602771s"
Jul 06 15:26:49.610163 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.571448    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.720268 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.671867    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.772411 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.772351    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.818625 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.818593    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-57.ec2.internal" not found
Jul 06 15:26:49.834305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.834251    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-57.ec2.internal" not found
Jul 06 15:26:49.845529 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.845513    2580 manager.go:337] "Starting Device Plugin manager"
Jul 06 15:26:49.845623 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.845547    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 15:26:49.845623 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.845557    2580 server.go:87] "Starting device plugin registration server"
Jul 06 15:26:49.845864 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.845852    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 15:26:49.845907 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.845878    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 15:26:49.846008 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.845992    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 15:26:49.846081 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.846069    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 15:26:49.846120 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.846081    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 15:26:49.846592 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.846571    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 15:26:49.846727 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:49.846622    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-57.ec2.internal\" not found"
Jul 06 15:26:49.878254 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.878223    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 15:26:49.893459 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.893432    2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-141-57.ec2.internal" not found
Jul 06 15:26:49.912655 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.912631    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 15:26:49.946372 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.946344    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 15:26:49.947591 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.947573    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 15:26:49.947728 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.947606    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 15:26:49.947728 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.947616    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeHasSufficientPID"
Jul 06 15:26:49.947728 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.947644    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-57.ec2.internal"
Jul 06 15:26:49.956089 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:49.956065    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.013757 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.013717    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 15:26:50.014994 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.014955    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 15:26:50.015133 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.015002    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 15:26:50.015133 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.015026    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 15:26:50.015133 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.015071    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 15:26:50.017264 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.017231    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 15:26:50.019884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.019863    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 15:26:50.115872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.115773    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"]
Jul 06 15:26:50.117519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.117500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.118606 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.118577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.142713 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.142670    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.147393 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.147376    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.162464 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.162436    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 15:26:50.170468 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.170445    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 15:26:50.176650 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.176624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77b865350d5ba6c456bb119a49e5b239-config\") pod \"kube-apiserver-proxy-ip-10-0-141-57.ec2.internal\" (UID: \"77b865350d5ba6c456bb119a49e5b239\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.176736 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.176657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8a04e126f7abf5397c90f27e9234ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal\" (UID: \"d8a04e126f7abf5397c90f27e9234ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.176736 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.176681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8a04e126f7abf5397c90f27e9234ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal\" (UID: \"d8a04e126f7abf5397c90f27e9234ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.276822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.276784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77b865350d5ba6c456bb119a49e5b239-config\") pod \"kube-apiserver-proxy-ip-10-0-141-57.ec2.internal\" (UID: \"77b865350d5ba6c456bb119a49e5b239\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.276822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.276817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8a04e126f7abf5397c90f27e9234ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal\" (UID: \"d8a04e126f7abf5397c90f27e9234ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.277034 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.276834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8a04e126f7abf5397c90f27e9234ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal\" (UID: \"d8a04e126f7abf5397c90f27e9234ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.277034 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.276893    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/77b865350d5ba6c456bb119a49e5b239-config\") pod \"kube-apiserver-proxy-ip-10-0-141-57.ec2.internal\" (UID: \"77b865350d5ba6c456bb119a49e5b239\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.277034 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.276943    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8a04e126f7abf5397c90f27e9234ece-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal\" (UID: \"d8a04e126f7abf5397c90f27e9234ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.277034 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.276975    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d8a04e126f7abf5397c90f27e9234ece-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal\" (UID: \"d8a04e126f7abf5397c90f27e9234ece\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.467434 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.467344    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.474107 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.474088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal"
Jul 06 15:26:50.547491 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.547450    2580 apiserver.go:52] "Watching apiserver"
Jul 06 15:26:50.553938 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.553900    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 15:26:50.555037 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.555011    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-sv76t","kube-system/global-pull-secret-syncer-5d7zj","kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl","openshift-cluster-node-tuning-operator/tuned-8chvh","openshift-dns/node-resolver-lf2hj","openshift-multus/multus-additional-cni-plugins-wvdqh","openshift-ovn-kubernetes/ovnkube-node-52rsz","kube-system/konnectivity-agent-xz86f","openshift-image-registry/node-ca-fkmm6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal","openshift-multus/multus-lxjwk","openshift-multus/network-metrics-daemon-rlzvq","openshift-network-diagnostics/network-check-target-ghq4d"]
Jul 06 15:26:50.556645 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.556622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.557813 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.557789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:50.557908 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.557867    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:26:50.558828 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.558803    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 15:26:50.558961 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.558941    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 15:26:50.558961 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.558957    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.559102 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.558948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.559102 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.559012    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 15:26:50.559102 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.559064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.559243 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.559149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4ktjr\""
Jul 06 15:26:50.560585 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.560304    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.561659 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.561635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 15:26:50.561774 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.561667    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.561774 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.561733    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.562710 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.561887    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rcvj7\""
Jul 06 15:26:50.562710 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.562082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.562710 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.562283    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.562896 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.562878    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lk6qz\""
Jul 06 15:26:50.562990 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.562957    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 15:26:50.563055 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.563039    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.563970 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.563953    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.564056 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.564019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-94q72\""
Jul 06 15:26:50.564056 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.564032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.564399 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.564382    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.565375 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.565363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.566040 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.566024    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.566117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.566026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dv65k\""
Jul 06 15:26:50.566117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.566030    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.566596 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.566581    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.567261 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567245    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 15:26:50.567351 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567246    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 15:26:50.567423 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567409    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.567476 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567423    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 15:26:50.567525 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567491    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 15:26:50.567785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567768    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xfjjd\""
Jul 06 15:26:50.567907 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567800    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 15:26:50.567907 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567889    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.567907 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.567891    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.568646 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.568628    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 15:26:50.568646 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.568640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9r2b6\""
Jul 06 15:26:50.568893 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.568880    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 15:26:50.569376 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.569361    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.569942 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.569926    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 15:26:50.569942 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.569935    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 15:26:50.570077 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.569926    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 15:26:50.570077 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.569983    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5s2vc\""
Jul 06 15:26:50.570774 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.570756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.570847 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.570828    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:26:50.571185 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.571170    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 15:26:50.571415 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.571402    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qqt8r\""
Jul 06 15:26:50.571999 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.571983    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:50.572074 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.572044    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:26:50.578817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysctl-conf\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.578937 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.578937 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-run-ovn-kubernetes\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.578937 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysconfig\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.578937 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-log-socket\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.578937 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.578987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-device-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579002    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-host-slash\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-modprobe-d\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-lib-modules\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-systemd-units\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovnkube-config\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbb51933-4cc0-4171-a916-676625b030ef-konnectivity-ca\") pod \"konnectivity-agent-xz86f\" (UID: \"fbb51933-4cc0-4171-a916-676625b030ef\") " pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-registration-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58rsl\" (UniqueName: \"kubernetes.io/projected/f2cf740b-9d20-43a3-a52c-d56a295fd62f-kube-api-access-58rsl\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcrjf\" (UniqueName: \"kubernetes.io/projected/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-kube-api-access-rcrjf\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579245    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-slash\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c2c1f01-a050-4040-a6c8-b39fd086a324-host\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pt9h\" (UniqueName: \"kubernetes.io/projected/4af80c0b-5938-4740-8c22-dfac3b02e1d0-kube-api-access-2pt9h\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfn86\" (UniqueName: \"kubernetes.io/projected/ab099878-8f06-4f6a-a7e0-aaabcc027145-kube-api-access-sfn86\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-cni-netd\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-systemd\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-host\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-var-lib-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-etc-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-cni-bin\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-systemd\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.579562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-sys\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-node-log\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-env-overrides\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9p5\" (UniqueName: \"kubernetes.io/projected/0c2c1f01-a050-4040-a6c8-b39fd086a324-kube-api-access-xn9p5\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-system-cni-dir\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-sys-fs\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-kubernetes\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysctl-d\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbb51933-4cc0-4171-a916-676625b030ef-agent-certs\") pod \"konnectivity-agent-xz86f\" (UID: \"fbb51933-4cc0-4171-a916-676625b030ef\") " pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-cnibin\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-os-release\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46j56\" (UniqueName: \"kubernetes.io/projected/9fd83a4d-6fff-4447-a374-13e06a499206-kube-api-access-46j56\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovnkube-script-lib\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c2c1f01-a050-4040-a6c8-b39fd086a324-serviceca\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-var-lib-kubelet\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4af80c0b-5938-4740-8c22-dfac3b02e1d0-hosts-file\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.579974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-kubelet\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh52z\" (UniqueName: \"kubernetes.io/projected/17725c45-8189-4c10-a419-29f80808d86a-kube-api-access-nh52z\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-socket-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-iptables-alerter-script\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-tuned\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4af80c0b-5938-4740-8c22-dfac3b02e1d0-tmp-dir\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-run-netns\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovn-node-metrics-cert\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-ovn\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89p4h\" (UniqueName: \"kubernetes.io/projected/0eabb18f-d646-4167-ad91-f6d61108bc5c-kube-api-access-89p4h\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-run\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab099878-8f06-4f6a-a7e0-aaabcc027145-tmp\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.580780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.580390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.610678 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.610633    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5qnz9"
Jul 06 15:26:50.622561 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.622530    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5qnz9"
Jul 06 15:26:50.668517 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.668490    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 15:26:50.680661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-slash\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.680786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c2c1f01-a050-4040-a6c8-b39fd086a324-host\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.680786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-conf-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.680786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pt9h\" (UniqueName: \"kubernetes.io/projected/4af80c0b-5938-4740-8c22-dfac3b02e1d0-kube-api-access-2pt9h\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.680786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfn86\" (UniqueName: \"kubernetes.io/projected/ab099878-8f06-4f6a-a7e0-aaabcc027145-kube-api-access-sfn86\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.680786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680769    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-slash\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.680786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680776    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0c2c1f01-a050-4040-a6c8-b39fd086a324-host\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-cni-netd\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43313d8c-0e8c-4bd0-8a8d-b72b64448225-cni-binary-copy\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680881    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-cni-netd\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-systemd\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-host\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-var-lib-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.680997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-cni-bin\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681012    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-systemd\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-etc-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681045    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-host\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-cni-bin\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681084    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-cni-bin\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-cni-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681110    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-var-lib-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-etc-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-k8s-cni-cncf-io\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-kubelet\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-systemd\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-sys\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-systemd\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-node-log\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681268    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-node-log\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681280    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-sys\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-env-overrides\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.681305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9p5\" (UniqueName: \"kubernetes.io/projected/0c2c1f01-a050-4040-a6c8-b39fd086a324-kube-api-access-xn9p5\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-system-cni-dir\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-sys-fs\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-kubernetes\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysctl-d\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbb51933-4cc0-4171-a916-676625b030ef-agent-certs\") pod \"konnectivity-agent-xz86f\" (UID: \"fbb51933-4cc0-4171-a916-676625b030ef\") " pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681406    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-system-cni-dir\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-cnibin\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681430    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-sys-fs\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-os-release\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681461    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-kubernetes\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46j56\" (UniqueName: \"kubernetes.io/projected/9fd83a4d-6fff-4447-a374-13e06a499206-kube-api-access-46j56\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681510    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-cnibin\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681517    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysctl-d\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681528    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681637    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-os-release\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovnkube-script-lib\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c2c1f01-a050-4040-a6c8-b39fd086a324-serviceca\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-system-cni-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681790    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-multus-certs\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-var-lib-kubelet\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681855    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-env-overrides\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4af80c0b-5938-4740-8c22-dfac3b02e1d0-hosts-file\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681928    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-var-lib-kubelet\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-kubelet\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.681998    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-kubelet\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682080    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4af80c0b-5938-4740-8c22-dfac3b02e1d0-hosts-file\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh52z\" (UniqueName: \"kubernetes.io/projected/17725c45-8189-4c10-a419-29f80808d86a-kube-api-access-nh52z\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-socket-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-iptables-alerter-script\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-tuned\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.682959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682429    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0c2c1f01-a050-4040-a6c8-b39fd086a324-serviceca\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682442    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4af80c0b-5938-4740-8c22-dfac3b02e1d0-tmp-dir\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-run-netns\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682509    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovnkube-script-lib\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovn-node-metrics-cert\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682566    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-run-netns\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682590    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-cnibin\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-ovn\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89p4h\" (UniqueName: \"kubernetes.io/projected/0eabb18f-d646-4167-ad91-f6d61108bc5c-kube-api-access-89p4h\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-socket-dir-parent\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-daemon-config\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-etc-kubernetes\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-run\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab099878-8f06-4f6a-a7e0-aaabcc027145-tmp\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.683770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7sn\" (UniqueName: \"kubernetes.io/projected/43313d8c-0e8c-4bd0-8a8d-b72b64448225-kube-api-access-2d7sn\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysctl-conf\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-run-ovn-kubernetes\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682973    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-iptables-alerter-script\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-os-release\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682986    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17725c45-8189-4c10-a419-29f80808d86a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-hostroot\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683056    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-ovn\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.683064    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.683143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:51.183108165 +0000 UTC m=+2.995970937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-run-openvswitch\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683363    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysctl-conf\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.682836    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4af80c0b-5938-4740-8c22-dfac3b02e1d0-tmp-dir\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683382    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-run\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysconfig\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.684523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-log-socket\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-device-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683528    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683561    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-socket-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/453f039d-a9cd-468d-88d5-34edaa9d8ef0-dbus\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683581    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-run-ovn-kubernetes\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-host-slash\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683609    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-modprobe-d\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-lib-modules\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683664    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-sysconfig\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683679    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-etc-selinux\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-systemd-units\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683736    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-log-socket\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.685315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovnkube-config\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683738    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0eabb18f-d646-4167-ad91-f6d61108bc5c-systemd-units\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbb51933-4cc0-4171-a916-676625b030ef-konnectivity-ca\") pod \"konnectivity-agent-xz86f\" (UID: \"fbb51933-4cc0-4171-a916-676625b030ef\") " pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-registration-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683794    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-device-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58rsl\" (UniqueName: \"kubernetes.io/projected/f2cf740b-9d20-43a3-a52c-d56a295fd62f-kube-api-access-58rsl\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683838    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-modprobe-d\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcrjf\" (UniqueName: \"kubernetes.io/projected/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-kube-api-access-rcrjf\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683858    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/453f039d-a9cd-468d-88d5-34edaa9d8ef0-kubelet-config\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683908    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2cf740b-9d20-43a3-a52c-d56a295fd62f-registration-dir\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683914    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab099878-8f06-4f6a-a7e0-aaabcc027145-lib-modules\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683951    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-host-slash\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.683914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.684337    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/17725c45-8189-4c10-a419-29f80808d86a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.684361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-netns\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.684409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-cni-multus\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.686198 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.685519    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovn-node-metrics-cert\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.686951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.685553    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab099878-8f06-4f6a-a7e0-aaabcc027145-etc-tuned\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.686951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.685663    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0eabb18f-d646-4167-ad91-f6d61108bc5c-ovnkube-config\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.686951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.685754    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fbb51933-4cc0-4171-a916-676625b030ef-konnectivity-ca\") pod \"konnectivity-agent-xz86f\" (UID: \"fbb51933-4cc0-4171-a916-676625b030ef\") " pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.686951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.685780    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fbb51933-4cc0-4171-a916-676625b030ef-agent-certs\") pod \"konnectivity-agent-xz86f\" (UID: \"fbb51933-4cc0-4171-a916-676625b030ef\") " pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.686951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.686901    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab099878-8f06-4f6a-a7e0-aaabcc027145-tmp\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.714768 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.714734    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58rsl\" (UniqueName: \"kubernetes.io/projected/f2cf740b-9d20-43a3-a52c-d56a295fd62f-kube-api-access-58rsl\") pod \"aws-ebs-csi-driver-node-8xwrl\" (UID: \"f2cf740b-9d20-43a3-a52c-d56a295fd62f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.719805 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.719731    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46j56\" (UniqueName: \"kubernetes.io/projected/9fd83a4d-6fff-4447-a374-13e06a499206-kube-api-access-46j56\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:50.721445 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.721418    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pt9h\" (UniqueName: \"kubernetes.io/projected/4af80c0b-5938-4740-8c22-dfac3b02e1d0-kube-api-access-2pt9h\") pod \"node-resolver-lf2hj\" (UID: \"4af80c0b-5938-4740-8c22-dfac3b02e1d0\") " pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.721553 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.721455    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcrjf\" (UniqueName: \"kubernetes.io/projected/5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3-kube-api-access-rcrjf\") pod \"iptables-alerter-sv76t\" (UID: \"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3\") " pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.724661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.724630    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh52z\" (UniqueName: \"kubernetes.io/projected/17725c45-8189-4c10-a419-29f80808d86a-kube-api-access-nh52z\") pod \"multus-additional-cni-plugins-wvdqh\" (UID: \"17725c45-8189-4c10-a419-29f80808d86a\") " pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.724874 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.724851    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89p4h\" (UniqueName: \"kubernetes.io/projected/0eabb18f-d646-4167-ad91-f6d61108bc5c-kube-api-access-89p4h\") pod \"ovnkube-node-52rsz\" (UID: \"0eabb18f-d646-4167-ad91-f6d61108bc5c\") " pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.725352 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.725329    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfn86\" (UniqueName: \"kubernetes.io/projected/ab099878-8f06-4f6a-a7e0-aaabcc027145-kube-api-access-sfn86\") pod \"tuned-8chvh\" (UID: \"ab099878-8f06-4f6a-a7e0-aaabcc027145\") " pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.725863 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.725846    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9p5\" (UniqueName: \"kubernetes.io/projected/0c2c1f01-a050-4040-a6c8-b39fd086a324-kube-api-access-xn9p5\") pod \"node-ca-fkmm6\" (UID: \"0c2c1f01-a050-4040-a6c8-b39fd086a324\") " pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.784931 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.784883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-system-cni-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.784931 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.784922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-multus-certs\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.784993    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-multus-certs\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-cnibin\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785026    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-system-cni-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-socket-dir-parent\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-daemon-config\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-cnibin\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-etc-kubernetes\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7sn\" (UniqueName: \"kubernetes.io/projected/43313d8c-0e8c-4bd0-8a8d-b72b64448225-kube-api-access-2d7sn\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785134 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785118    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-socket-dir-parent\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-os-release\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785146    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-etc-kubernetes\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-hostroot\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/453f039d-a9cd-468d-88d5-34edaa9d8ef0-dbus\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/453f039d-a9cd-468d-88d5-34edaa9d8ef0-kubelet-config\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-netns\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-cni-multus\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785296    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-cni-multus\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785333    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-netns\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785369    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-os-release\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-hostroot\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785398    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/453f039d-a9cd-468d-88d5-34edaa9d8ef0-dbus\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785407    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/453f039d-a9cd-468d-88d5-34edaa9d8ef0-kubelet-config\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.785444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-conf-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43313d8c-0e8c-4bd0-8a8d-b72b64448225-cni-binary-copy\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785507    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-conf-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-cni-bin\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-cni-bin\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-cni-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-k8s-cni-cncf-io\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-kubelet\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785655    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-run-k8s-cni-cncf-io\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785662    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-cni-dir\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785663    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43313d8c-0e8c-4bd0-8a8d-b72b64448225-host-var-lib-kubelet\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.785741    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43313d8c-0e8c-4bd0-8a8d-b72b64448225-multus-daemon-config\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786076 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.786044    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:50.786659 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.786266    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43313d8c-0e8c-4bd0-8a8d-b72b64448225-cni-binary-copy\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.786659 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.786300    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:51.286271447 +0000 UTC m=+3.099134204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:50.792432 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.792407    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:26:50.792432 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.792431    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:26:50.792432 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.792441    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:50.792632 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:50.792507    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:26:51.292493058 +0000 UTC m=+3.105355813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:50.794934 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.794903    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7sn\" (UniqueName: \"kubernetes.io/projected/43313d8c-0e8c-4bd0-8a8d-b72b64448225-kube-api-access-2d7sn\") pod \"multus-lxjwk\" (UID: \"43313d8c-0e8c-4bd0-8a8d-b72b64448225\") " pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.886497 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.886441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wvdqh"
Jul 06 15:26:50.901072 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.901040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl"
Jul 06 15:26:50.908959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.908930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sv76t"
Jul 06 15:26:50.921130 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.921102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8chvh"
Jul 06 15:26:50.936347 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.936317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lf2hj"
Jul 06 15:26:50.942331 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.942303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:26:50.958491 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.958461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:26:50.978873 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.978809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fkmm6"
Jul 06 15:26:50.986051 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:50.986025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lxjwk"
Jul 06 15:26:50.998627 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:50.998589    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a04e126f7abf5397c90f27e9234ece.slice/crio-2ccec30b9449d207c8a9d688b623b1158fbb5b851ace706356f765e5ebf5328e WatchSource:0}: Error finding container 2ccec30b9449d207c8a9d688b623b1158fbb5b851ace706356f765e5ebf5328e: Status 404 returned error can't find the container with id 2ccec30b9449d207c8a9d688b623b1158fbb5b851ace706356f765e5ebf5328e
Jul 06 15:26:50.998982 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:50.998960    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b865350d5ba6c456bb119a49e5b239.slice/crio-9962e3100863df83ae713109232d04639b90807fba24e0a6b3903c0d06146d79 WatchSource:0}: Error finding container 9962e3100863df83ae713109232d04639b90807fba24e0a6b3903c0d06146d79: Status 404 returned error can't find the container with id 9962e3100863df83ae713109232d04639b90807fba24e0a6b3903c0d06146d79
Jul 06 15:26:50.999714 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:50.999674    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb51933_4cc0_4171_a916_676625b030ef.slice/crio-421c32398ff6843905933f4ba19d10a83eb06d4d6320326909a8f635129b668b WatchSource:0}: Error finding container 421c32398ff6843905933f4ba19d10a83eb06d4d6320326909a8f635129b668b: Status 404 returned error can't find the container with id 421c32398ff6843905933f4ba19d10a83eb06d4d6320326909a8f635129b668b
Jul 06 15:26:51.001154 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.001134    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2c1f01_a050_4040_a6c8_b39fd086a324.slice/crio-ab365b5611ec6445953ed843807037bcd8c994625cf92206982b3a6a7631c77c WatchSource:0}: Error finding container ab365b5611ec6445953ed843807037bcd8c994625cf92206982b3a6a7631c77c: Status 404 returned error can't find the container with id ab365b5611ec6445953ed843807037bcd8c994625cf92206982b3a6a7631c77c
Jul 06 15:26:51.001644 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.001622    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43313d8c_0e8c_4bd0_8a8d_b72b64448225.slice/crio-4333da2b9dd2b6606bbb8d99f7ed5899e9d04a2d6d614b8bf647a0550e26c3d5 WatchSource:0}: Error finding container 4333da2b9dd2b6606bbb8d99f7ed5899e9d04a2d6d614b8bf647a0550e26c3d5: Status 404 returned error can't find the container with id 4333da2b9dd2b6606bbb8d99f7ed5899e9d04a2d6d614b8bf647a0550e26c3d5
Jul 06 15:26:51.003906 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.003855    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 15:26:51.019048 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.018993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lxjwk" event={"ID":"43313d8c-0e8c-4bd0-8a8d-b72b64448225","Type":"ContainerStarted","Data":"4333da2b9dd2b6606bbb8d99f7ed5899e9d04a2d6d614b8bf647a0550e26c3d5"}
Jul 06 15:26:51.020064 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.020034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal" event={"ID":"d8a04e126f7abf5397c90f27e9234ece","Type":"ContainerStarted","Data":"2ccec30b9449d207c8a9d688b623b1158fbb5b851ace706356f765e5ebf5328e"}
Jul 06 15:26:51.021087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.021059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fkmm6" event={"ID":"0c2c1f01-a050-4040-a6c8-b39fd086a324","Type":"ContainerStarted","Data":"ab365b5611ec6445953ed843807037bcd8c994625cf92206982b3a6a7631c77c"}
Jul 06 15:26:51.022122 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.022101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xz86f" event={"ID":"fbb51933-4cc0-4171-a916-676625b030ef","Type":"ContainerStarted","Data":"421c32398ff6843905933f4ba19d10a83eb06d4d6320326909a8f635129b668b"}
Jul 06 15:26:51.024562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.024163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal" event={"ID":"77b865350d5ba6c456bb119a49e5b239","Type":"ContainerStarted","Data":"9962e3100863df83ae713109232d04639b90807fba24e0a6b3903c0d06146d79"}
Jul 06 15:26:51.188262 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.188229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:51.188490 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.188350    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:51.188490 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.188404    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:52.188388935 +0000 UTC m=+4.001251690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:51.232108 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.232071    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2b11bc_0d84_4eb7_ae98_a01f159b9cf3.slice/crio-5b2cf92301dec28aab1154d4004e3c21d8fbaaad8829ee31238c4ccc90b011b6 WatchSource:0}: Error finding container 5b2cf92301dec28aab1154d4004e3c21d8fbaaad8829ee31238c4ccc90b011b6: Status 404 returned error can't find the container with id 5b2cf92301dec28aab1154d4004e3c21d8fbaaad8829ee31238c4ccc90b011b6
Jul 06 15:26:51.234581 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.233971    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eabb18f_d646_4167_ad91_f6d61108bc5c.slice/crio-03837d96f2a939440ad42afcf1f9fffacb78906f3bf0797ce93ccc8688141ba5 WatchSource:0}: Error finding container 03837d96f2a939440ad42afcf1f9fffacb78906f3bf0797ce93ccc8688141ba5: Status 404 returned error can't find the container with id 03837d96f2a939440ad42afcf1f9fffacb78906f3bf0797ce93ccc8688141ba5
Jul 06 15:26:51.257312 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.257281    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af80c0b_5938_4740_8c22_dfac3b02e1d0.slice/crio-0ce30cc941db11148a8db28c806e2961e9ef61601db600553029b89dfc3b8d9c WatchSource:0}: Error finding container 0ce30cc941db11148a8db28c806e2961e9ef61601db600553029b89dfc3b8d9c: Status 404 returned error can't find the container with id 0ce30cc941db11148a8db28c806e2961e9ef61601db600553029b89dfc3b8d9c
Jul 06 15:26:51.263805 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.263778    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17725c45_8189_4c10_a419_29f80808d86a.slice/crio-4bffcf9f8d971cb315ad295022e6c8952c0a70f143ed85e73373362dab68d218 WatchSource:0}: Error finding container 4bffcf9f8d971cb315ad295022e6c8952c0a70f143ed85e73373362dab68d218: Status 404 returned error can't find the container with id 4bffcf9f8d971cb315ad295022e6c8952c0a70f143ed85e73373362dab68d218
Jul 06 15:26:51.289211 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.289180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:51.289373 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.289326    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:51.289414 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.289389    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:52.28937341 +0000 UTC m=+4.102236166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:51.312924 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.312885    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2cf740b_9d20_43a3_a52c_d56a295fd62f.slice/crio-35c3ecc8b6db9a958f731683b86f78969201e9fe0d83bcc369d1dce8ca66016c WatchSource:0}: Error finding container 35c3ecc8b6db9a958f731683b86f78969201e9fe0d83bcc369d1dce8ca66016c: Status 404 returned error can't find the container with id 35c3ecc8b6db9a958f731683b86f78969201e9fe0d83bcc369d1dce8ca66016c
Jul 06 15:26:51.354728 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:26:51.354678    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab099878_8f06_4f6a_a7e0_aaabcc027145.slice/crio-0e8481218e29bad564c3f347461c7affbbd1fd779b306be23b3f142a62432748 WatchSource:0}: Error finding container 0e8481218e29bad564c3f347461c7affbbd1fd779b306be23b3f142a62432748: Status 404 returned error can't find the container with id 0e8481218e29bad564c3f347461c7affbbd1fd779b306be23b3f142a62432748
Jul 06 15:26:51.390428 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.390376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:51.390624 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.390595    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:26:51.390731 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.390634    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:26:51.390731 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.390649    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:51.390841 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:51.390731    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:26:52.390694549 +0000 UTC m=+4.203557311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:51.626284 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.626182    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 15:21:50 +0000 UTC" deadline="2028-02-10 05:18:27.807818487 +0000 UTC"
Jul 06 15:26:51.626284 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:51.626224    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14005h51m36.181599071s"
Jul 06 15:26:52.027308 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.025779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:52.027308 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.025986    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:26:52.027308 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.026566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:52.027308 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.026830    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:26:52.027308 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.026972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:52.027308 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.027086    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:26:52.070163 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.070121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl" event={"ID":"f2cf740b-9d20-43a3-a52c-d56a295fd62f","Type":"ContainerStarted","Data":"35c3ecc8b6db9a958f731683b86f78969201e9fe0d83bcc369d1dce8ca66016c"}
Jul 06 15:26:52.094923 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.094877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerStarted","Data":"4bffcf9f8d971cb315ad295022e6c8952c0a70f143ed85e73373362dab68d218"}
Jul 06 15:26:52.113045 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.113002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"03837d96f2a939440ad42afcf1f9fffacb78906f3bf0797ce93ccc8688141ba5"}
Jul 06 15:26:52.130360 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.130314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sv76t" event={"ID":"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3","Type":"ContainerStarted","Data":"5b2cf92301dec28aab1154d4004e3c21d8fbaaad8829ee31238c4ccc90b011b6"}
Jul 06 15:26:52.142372 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.142288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8chvh" event={"ID":"ab099878-8f06-4f6a-a7e0-aaabcc027145","Type":"ContainerStarted","Data":"0e8481218e29bad564c3f347461c7affbbd1fd779b306be23b3f142a62432748"}
Jul 06 15:26:52.149146 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.149108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lf2hj" event={"ID":"4af80c0b-5938-4740-8c22-dfac3b02e1d0","Type":"ContainerStarted","Data":"0ce30cc941db11148a8db28c806e2961e9ef61601db600553029b89dfc3b8d9c"}
Jul 06 15:26:52.199132 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.199091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:52.199440 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.199421    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:52.199503 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.199497    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:54.199475081 +0000 UTC m=+6.012337844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:52.300242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.300153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:52.300415 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.300353    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:52.300483 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.300417    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:54.300397904 +0000 UTC m=+6.113260676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:52.401478 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.401438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:52.401665 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.401628    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:26:52.401665 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.401653    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:26:52.401665 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.401667    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:52.401881 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:52.401738    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:26:54.401718611 +0000 UTC m=+6.214581375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:52.626996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.626874    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 15:21:50 +0000 UTC" deadline="2028-01-13 09:46:12.215310102 +0000 UTC"
Jul 06 15:26:52.626996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:52.626941    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13338h19m19.58837331s"
Jul 06 15:26:54.022215 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:54.020927    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:54.022215 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.021309    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:26:54.022215 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:54.021679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:54.022215 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:54.022076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:54.022215 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.022170    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:26:54.023691 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.023612    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:26:54.223899 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:54.223861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:54.224097 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.224078    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:54.224161 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.224150    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:58.22413065 +0000 UTC m=+10.036993422 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:54.324610 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:54.324521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:54.324795 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.324693    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:54.324795 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.324772    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:26:58.324750904 +0000 UTC m=+10.137613660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:54.424944 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:54.424911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:54.425123 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.425049    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:26:54.425123 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.425065    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:26:54.425123 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.425074    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:54.425123 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:54.425111    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:26:58.425099447 +0000 UTC m=+10.237962202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:56.019373 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:56.019339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:56.019888 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:56.019483    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:26:56.019888 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:56.019869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:56.019997 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:56.019966    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:26:56.020055 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:56.020013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:56.020106 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:56.020078    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:26:58.015863 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:58.015813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:58.016343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:58.015813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:58.016343 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.015972    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:26:58.016343 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.016074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:26:58.016343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:58.015813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:58.016343 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.016165    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:26:58.259266 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:58.259221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:26:58.259460 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.259381    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:58.259460 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.259449    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:27:06.259431818 +0000 UTC m=+18.072294574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:26:58.359730 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:58.359626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:26:58.359924 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.359790    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:58.359924 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.359867    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:27:06.359835718 +0000 UTC m=+18.172698476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:26:58.460432 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:26:58.460393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:26:58.460620 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.460568    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:26:58.460620 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.460595    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:26:58.460620 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.460620    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:26:58.460804 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:26:58.460689    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:27:06.460672231 +0000 UTC m=+18.273535008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:27:00.021593 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:00.021552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:00.022044 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:00.021720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:00.022118 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:00.022076    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:00.022485 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:00.022436    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:00.022554 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:00.022528    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:00.022678 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:00.022652    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:02.015258 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:02.015218    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:02.015751 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:02.015218    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:02.015751 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:02.015357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:02.015751 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:02.015355    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:02.015751 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:02.015500    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:02.015751 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:02.015592    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:04.015924 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:04.015477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:04.015924 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:04.015481    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:04.015924 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:04.015603    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:04.015924 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:04.015477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:04.015924 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:04.015670    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:04.015924 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:04.015799    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:06.015243 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:06.015198    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:06.015243 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:06.015243    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:06.015856 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:06.015210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:06.015856 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.015321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:06.015856 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.015391    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:06.015856 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.015494    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:06.319710 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:06.319598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:06.319901 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.319776    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:27:06.319901 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.319871    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:27:22.319850655 +0000 UTC m=+34.132713414 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:27:06.420938 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:06.420903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:06.421103 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.421080    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:27:06.421179 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.421166    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:27:22.42114357 +0000 UTC m=+34.234006345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:27:06.521812 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:06.521770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:06.521974 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.521919    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:27:06.521974 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.521941    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:27:06.521974 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.521952    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:27:06.522138 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:06.522015    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:27:22.521997576 +0000 UTC m=+34.334860346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:27:08.016205 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:08.016159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:08.016781 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:08.016159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:08.016781 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:08.016288    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:08.016781 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:08.016306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:08.016781 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:08.016391    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:08.016781 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:08.016471    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:10.016499 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:10.016458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:10.017001 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:10.016596    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:10.017001 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:10.016600    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:10.017001 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:10.016629    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:10.017001 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:10.016681    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:10.017214 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:10.017117    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:12.018784 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.018522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:12.019185 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.018566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:12.019185 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:12.018875    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:12.019185 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.018566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:12.019185 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:12.018933    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:12.019185 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:12.019030    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:12.198452 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.198363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal" event={"ID":"77b865350d5ba6c456bb119a49e5b239","Type":"ContainerStarted","Data":"718aa49a657dfca7fe7985aeef33224ada87bf4afa8bcb3d9ed33d904cc25ec1"}
Jul 06 15:27:12.201003 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.200981    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:27:12.201471 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.201297    2580 generic.go:356] "Generic (PLEG): container finished" podID="0eabb18f-d646-4167-ad91-f6d61108bc5c" containerID="aa86ac48bb030d189e4dbb41930d7d166f51d9fea281317fdd7af6d80c978a3c" exitCode=1
Jul 06 15:27:12.201561 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.201371    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"65bf2a58481e388900823f190d35d6a54c5a350f4a852e827b6122b996a50c63"}
Jul 06 15:27:12.201644 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.201629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerDied","Data":"aa86ac48bb030d189e4dbb41930d7d166f51d9fea281317fdd7af6d80c978a3c"}
Jul 06 15:27:12.201694 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.201675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"5f8ef7eb0c368c74651155169e78ae64fc6d713c492691e8bf55669ca0632202"}
Jul 06 15:27:12.206859 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.206824    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lxjwk" event={"ID":"43313d8c-0e8c-4bd0-8a8d-b72b64448225","Type":"ContainerStarted","Data":"627122d7ed74d44cded01065f44edfd62a2f6ae0adfb31f04fb862c62030eb42"}
Jul 06 15:27:12.208578 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.208550    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8chvh" event={"ID":"ab099878-8f06-4f6a-a7e0-aaabcc027145","Type":"ContainerStarted","Data":"e6deb7db16a136f93ed4723e7338e09d0d921d87537df4b8dc7b1113c51efd28"}
Jul 06 15:27:12.214614 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.214552    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-57.ec2.internal" podStartSLOduration=22.214536357 podStartE2EDuration="22.214536357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:12.213898343 +0000 UTC m=+24.026761121" watchObservedRunningTime="2026-07-06 15:27:12.214536357 +0000 UTC m=+24.027399134"
Jul 06 15:27:12.233471 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.232616    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8chvh" podStartSLOduration=2.613377146 podStartE2EDuration="22.232595151s" totalImagesPullingTime="19.619218005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:12.231950705 +0000 UTC m=+24.044813485" watchObservedRunningTime="2026-07-06 15:27:12.232595151 +0000 UTC m=+24.045457929"
Jul 06 15:27:12.250487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:12.250430    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lxjwk" podStartSLOduration=2.261519599 podStartE2EDuration="22.250411032s" totalImagesPullingTime="19.988891433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:12.250200182 +0000 UTC m=+24.063062959" watchObservedRunningTime="2026-07-06 15:27:12.250411032 +0000 UTC m=+24.063273811"
Jul 06 15:27:13.213304 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.213188    2580 generic.go:356] "Generic (PLEG): container finished" podID="17725c45-8189-4c10-a419-29f80808d86a" containerID="5f9dd78d93e48496ab1e1e4e56359d827956f7c25aa8c01e5f7b2afde9a219e7" exitCode=0
Jul 06 15:27:13.213304 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.213230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerDied","Data":"5f9dd78d93e48496ab1e1e4e56359d827956f7c25aa8c01e5f7b2afde9a219e7"}
Jul 06 15:27:13.216486 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.216472    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:27:13.216960 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.216933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"2630d918f9943ec8ed307f6ee7b74820af1d2c744c70b1f2cf0ca5a418b30659"}
Jul 06 15:27:13.217071 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.217026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"1dbd285a3be05752799ee98182aa35d42accc8ccca2248bbfcc751fbefb752e4"}
Jul 06 15:27:13.217130 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.217080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"a5443ebfc0738557e6cb5c610057da427fef4caa389e27144ddae94fffc49aaf"}
Jul 06 15:27:13.218537 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.218507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sv76t" event={"ID":"5f2b11bc-0d84-4eb7-ae98-a01f159b9cf3","Type":"ContainerStarted","Data":"82092bcc4b48e36e6fe912ea73bcadce9c2d4055467d99941bc5b1d0e0b9c63f"}
Jul 06 15:27:13.220470 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.220445    2580 generic.go:356] "Generic (PLEG): container finished" podID="d8a04e126f7abf5397c90f27e9234ece" containerID="a7bbeec00a8960d6626f03f971713bcb0e964f2f75f3674fafdbdf121517535f" exitCode=0
Jul 06 15:27:13.220579 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.220530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal" event={"ID":"d8a04e126f7abf5397c90f27e9234ece","Type":"ContainerDied","Data":"a7bbeec00a8960d6626f03f971713bcb0e964f2f75f3674fafdbdf121517535f"}
Jul 06 15:27:13.222323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.222296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lf2hj" event={"ID":"4af80c0b-5938-4740-8c22-dfac3b02e1d0","Type":"ContainerStarted","Data":"7f8ec371ef4bbfddca0112a86dd6996a7800d2dd20f5f2bad155f58cb5fa4f21"}
Jul 06 15:27:13.223805 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.223784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fkmm6" event={"ID":"0c2c1f01-a050-4040-a6c8-b39fd086a324","Type":"ContainerStarted","Data":"d0320f6c2f0d461da937f74a11dde6de9cd9fe0af400773a28a29204f3a69eb1"}
Jul 06 15:27:13.225346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.225323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xz86f" event={"ID":"fbb51933-4cc0-4171-a916-676625b030ef","Type":"ContainerStarted","Data":"f8195fbbfd49e7a79c72058347a9ccbd075a5e87fd6098e3f0993e3daa089b5c"}
Jul 06 15:27:13.226846 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.226823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl" event={"ID":"f2cf740b-9d20-43a3-a52c-d56a295fd62f","Type":"ContainerStarted","Data":"33f78b094e0d92979995b047079e7e2d25dfff260d38345d7a64699c96c152c5"}
Jul 06 15:27:13.266600 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.266545    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lf2hj" podStartSLOduration=3.543172892 podStartE2EDuration="23.266528115s" totalImagesPullingTime="19.723355223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:13.250872704 +0000 UTC m=+25.063735485" watchObservedRunningTime="2026-07-06 15:27:13.266528115 +0000 UTC m=+25.079390891"
Jul 06 15:27:13.266926 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.266893    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xz86f" podStartSLOduration=3.294393016 podStartE2EDuration="23.26688231s" totalImagesPullingTime="19.972489294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:13.266306627 +0000 UTC m=+25.079169403" watchObservedRunningTime="2026-07-06 15:27:13.26688231 +0000 UTC m=+25.079745098"
Jul 06 15:27:13.280691 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.280584    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fkmm6" podStartSLOduration=3.307516878 podStartE2EDuration="23.280562857s" totalImagesPullingTime="19.973045979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:13.279716563 +0000 UTC m=+25.092579353" watchObservedRunningTime="2026-07-06 15:27:13.280562857 +0000 UTC m=+25.093425635"
Jul 06 15:27:13.299572 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.299517    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sv76t" podStartSLOduration=3.562402371 podStartE2EDuration="23.29949709s" totalImagesPullingTime="19.737094719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:13.299246981 +0000 UTC m=+25.112109760" watchObservedRunningTime="2026-07-06 15:27:13.29949709 +0000 UTC m=+25.112359869"
Jul 06 15:27:13.660261 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.660225    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:27:13.660908 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.660883    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:27:13.725020 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.724985    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 15:27:13.860322 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.860139    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T15:27:13.725006417Z","UUID":"e6631e91-5a99-43a3-826c-947441f76bee","Handler":null,"Name":"","Endpoint":""}
Jul 06 15:27:13.861959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.861934    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 15:27:13.862092 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:13.861988    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 15:27:14.016188 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.016151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:14.016376 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.016156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:14.016376 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:14.016301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:14.016376 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.016337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:14.016517 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:14.016423    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:14.016517 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:14.016489    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:14.233005 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.232915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl" event={"ID":"f2cf740b-9d20-43a3-a52c-d56a295fd62f","Type":"ContainerStarted","Data":"00e7807c8e48f6834daa4f2c998b679631bdc2dde8f8d345aa55f8adb3ce1609"}
Jul 06 15:27:14.235847 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.235810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal" event={"ID":"d8a04e126f7abf5397c90f27e9234ece","Type":"ContainerStarted","Data":"9c3aa09c5fd120afa727e5c60f076a9999d3819dbb6fadd0ae9e1552752c6957"}
Jul 06 15:27:14.236413 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.236278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:27:14.237158 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.237134    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xz86f"
Jul 06 15:27:14.256756 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:14.256673    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-57.ec2.internal" podStartSLOduration=24.256650886 podStartE2EDuration="24.256650886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:14.255929568 +0000 UTC m=+26.068792357" watchObservedRunningTime="2026-07-06 15:27:14.256650886 +0000 UTC m=+26.069513663"
Jul 06 15:27:15.241186 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:15.240869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl" event={"ID":"f2cf740b-9d20-43a3-a52c-d56a295fd62f","Type":"ContainerStarted","Data":"bca8e7902dad74431e5823c8d1ece9efb0ab6a193167b734908ac09d9916c4b1"}
Jul 06 15:27:15.244880 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:15.244848    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:27:15.245423 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:15.245388    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"8e26521dc1e3ce1df0dc3b42661723229c0238814f5d0a8f06fe16a19c537401"}
Jul 06 15:27:15.267580 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:15.267523    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8xwrl" podStartSLOduration=3.2878304480000002 podStartE2EDuration="25.267505235s" totalImagesPullingTime="21.979674787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:15.266245881 +0000 UTC m=+27.079108673" watchObservedRunningTime="2026-07-06 15:27:15.267505235 +0000 UTC m=+27.080368013"
Jul 06 15:27:16.015760 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:16.015692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:16.015760 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:16.015751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:16.016250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:16.015751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:16.016250 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:16.015829    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:16.016250 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:16.015953    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:16.016250 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:16.016036    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:18.016054 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.015831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:18.016859 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.015831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:18.016859 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:18.016154    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:18.016859 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.015831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:18.016859 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:18.016207    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:18.016859 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:18.016282    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:18.253725 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.253673    2580 generic.go:356] "Generic (PLEG): container finished" podID="17725c45-8189-4c10-a419-29f80808d86a" containerID="92cadf692f50b45cf7c09892c3166b2333d7375d336013dd27cfa8378e86b2bf" exitCode=0
Jul 06 15:27:18.253929 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.253784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerDied","Data":"92cadf692f50b45cf7c09892c3166b2333d7375d336013dd27cfa8378e86b2bf"}
Jul 06 15:27:18.257310 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.257293    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:27:18.257632 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.257605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"baf9fc2c8ee7ad14a549199fa59abe93c399bd01c06e70631625c91a546b54ea"}
Jul 06 15:27:18.257993 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.257977    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:27:18.258083 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.258062    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:27:18.258135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.258102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:27:18.258186 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.258146    2580 scope.go:117] "RemoveContainer" containerID="aa86ac48bb030d189e4dbb41930d7d166f51d9fea281317fdd7af6d80c978a3c"
Jul 06 15:27:18.275878 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.275840    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:27:18.276291 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:18.276273    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:27:19.263951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.263917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:27:19.264423 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.264267    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" event={"ID":"0eabb18f-d646-4167-ad91-f6d61108bc5c","Type":"ContainerStarted","Data":"f8c0200332f21487d56a5a597cbaf5c0f562c9ae7e69b90361bbb1e0a07c467d"}
Jul 06 15:27:19.295917 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.295851    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz" podStartSLOduration=8.727672124 podStartE2EDuration="29.295830379s" totalImagesPullingTime="20.568158255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:19.295068305 +0000 UTC m=+31.107931124" watchObservedRunningTime="2026-07-06 15:27:19.295830379 +0000 UTC m=+31.108693159"
Jul 06 15:27:19.301519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.301488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rlzvq"]
Jul 06 15:27:19.301661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.301634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:19.301809 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:19.301779    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:19.304241 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.303994    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ghq4d"]
Jul 06 15:27:19.304241 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.304142    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:19.304241 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:19.304238    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:19.304574 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.304557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5d7zj"]
Jul 06 15:27:19.304650 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:19.304639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:19.304746 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:19.304727    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:20.268942 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:20.268748    2580 generic.go:356] "Generic (PLEG): container finished" podID="17725c45-8189-4c10-a419-29f80808d86a" containerID="dd99eb64299c97a3aca1676496448d05b10215a0337dab9bf7e78a490428adc6" exitCode=0
Jul 06 15:27:20.269365 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:20.268836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerDied","Data":"dd99eb64299c97a3aca1676496448d05b10215a0337dab9bf7e78a490428adc6"}
Jul 06 15:27:21.015611 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:21.015570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:21.015611 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:21.015603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:21.015846 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:21.015613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:21.015846 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:21.015713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:21.015846 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:21.015753    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:21.015846 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:21.015797    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:22.276475 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:22.276438    2580 generic.go:356] "Generic (PLEG): container finished" podID="17725c45-8189-4c10-a419-29f80808d86a" containerID="c670222690f4ddaf8842f48572044d9c796873452d888d709af3d4ee2cb31d4e" exitCode=0
Jul 06 15:27:22.277130 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:22.276507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerDied","Data":"c670222690f4ddaf8842f48572044d9c796873452d888d709af3d4ee2cb31d4e"}
Jul 06 15:27:22.337003 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:22.336960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:22.337210 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.337153    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:27:22.337268 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.337235    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:27:54.337211632 +0000 UTC m=+66.150074387 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 15:27:22.437373 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:22.437330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:22.437602 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.437577    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 15:27:22.437668 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.437655    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret podName:453f039d-a9cd-468d-88d5-34edaa9d8ef0 nodeName:}" failed. No retries permitted until 2026-07-06 15:27:54.43763374 +0000 UTC m=+66.250496495 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret") pod "global-pull-secret-syncer-5d7zj" (UID: "453f039d-a9cd-468d-88d5-34edaa9d8ef0") : object "kube-system"/"original-pull-secret" not registered
Jul 06 15:27:22.538468 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:22.538382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:22.538610 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.538502    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 15:27:22.538610 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.538517    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 15:27:22.538610 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.538525    2580 projected.go:196] Error preparing data for projected volume kube-api-access-2694d for pod openshift-network-diagnostics/network-check-target-ghq4d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:27:22.538610 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:22.538574    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d podName:dafdb3cb-d443-4b56-a972-f9c73edae23d nodeName:}" failed. No retries permitted until 2026-07-06 15:27:54.538559158 +0000 UTC m=+66.351421914 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2694d" (UniqueName: "kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d") pod "network-check-target-ghq4d" (UID: "dafdb3cb-d443-4b56-a972-f9c73edae23d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 15:27:23.016202 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:23.016168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:23.016202 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:23.016188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:23.016434 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:23.016168    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:23.016434 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:23.016284    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ghq4d" podUID="dafdb3cb-d443-4b56-a972-f9c73edae23d"
Jul 06 15:27:23.016434 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:23.016380    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rlzvq" podUID="9fd83a4d-6fff-4447-a374-13e06a499206"
Jul 06 15:27:23.016565 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:23.016462    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5d7zj" podUID="453f039d-a9cd-468d-88d5-34edaa9d8ef0"
Jul 06 15:27:24.465073 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.465043    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-57.ec2.internal" event="NodeReady"
Jul 06 15:27:24.465580 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.465209    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 15:27:24.542186 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.542152    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5t8j5"]
Jul 06 15:27:24.544403 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.544374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.547655 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.547622    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r8n47"]
Jul 06 15:27:24.547824 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.547678    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 15:27:24.547824 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.547692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 15:27:24.548342 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.548197    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6q4fd\""
Jul 06 15:27:24.548342 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.548247    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 15:27:24.549647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.549625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.554310 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.554082    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 15:27:24.554310 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.554178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 15:27:24.554310 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.554194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cwwg8\""
Jul 06 15:27:24.561360 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.561328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5t8j5"]
Jul 06 15:27:24.565792 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.565762    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8n47"]
Jul 06 15:27:24.650885 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.650834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-tmp-dir\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.651059 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.650899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqshv\" (UniqueName: \"kubernetes.io/projected/7b625305-ec07-489b-9d17-6b0a5262a856-kube-api-access-dqshv\") pod \"ingress-canary-5t8j5\" (UID: \"7b625305-ec07-489b-9d17-6b0a5262a856\") " pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.651059 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.650973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.651059 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.651041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b625305-ec07-489b-9d17-6b0a5262a856-cert\") pod \"ingress-canary-5t8j5\" (UID: \"7b625305-ec07-489b-9d17-6b0a5262a856\") " pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.651158 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.651061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-config-volume\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.651158 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.651080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6prn6\" (UniqueName: \"kubernetes.io/projected/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-kube-api-access-6prn6\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.752235 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-tmp-dir\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.752235 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqshv\" (UniqueName: \"kubernetes.io/projected/7b625305-ec07-489b-9d17-6b0a5262a856-kube-api-access-dqshv\") pod \"ingress-canary-5t8j5\" (UID: \"7b625305-ec07-489b-9d17-6b0a5262a856\") " pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.752235 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.752476 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b625305-ec07-489b-9d17-6b0a5262a856-cert\") pod \"ingress-canary-5t8j5\" (UID: \"7b625305-ec07-489b-9d17-6b0a5262a856\") " pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.752476 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752303    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-config-volume\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.752476 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:24.752438    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:24.752562 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:24.752498    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:27:25.252481759 +0000 UTC m=+37.065344515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:27:24.752562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752494    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-tmp-dir\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.752562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6prn6\" (UniqueName: \"kubernetes.io/projected/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-kube-api-access-6prn6\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.753013 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.752996    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-config-volume\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.757495 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.757318    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b625305-ec07-489b-9d17-6b0a5262a856-cert\") pod \"ingress-canary-5t8j5\" (UID: \"7b625305-ec07-489b-9d17-6b0a5262a856\") " pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.764002 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.763971    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6prn6\" (UniqueName: \"kubernetes.io/projected/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-kube-api-access-6prn6\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:24.765995 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.765970    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqshv\" (UniqueName: \"kubernetes.io/projected/7b625305-ec07-489b-9d17-6b0a5262a856-kube-api-access-dqshv\") pod \"ingress-canary-5t8j5\" (UID: \"7b625305-ec07-489b-9d17-6b0a5262a856\") " pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:24.857646 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:24.857612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5t8j5"
Jul 06 15:27:25.015726 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.015577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:25.015726 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.015577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:25.015964 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.015577    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:25.019218 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.018975    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 15:27:25.019218 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.018975    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5xpkj\""
Jul 06 15:27:25.019218 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.019020    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 15:27:25.019218 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.019036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 15:27:25.019604 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.019482    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 15:27:25.019676 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.019660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-km2tx\""
Jul 06 15:27:25.024089 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.024064    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5t8j5"]
Jul 06 15:27:25.257211 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:25.257170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:25.257414 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:25.257299    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:25.257414 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:25.257397    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:27:26.257364941 +0000 UTC m=+38.070227710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:27:26.267237 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:26.267200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:26.267828 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:26.267351    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:26.267828 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:26.267427    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:27:28.267408649 +0000 UTC m=+40.080271409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:27:26.448608 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:27:26.448569    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b625305_ec07_489b_9d17_6b0a5262a856.slice/crio-233a568f01da311e7b08fb0408ec6f79acff9e913acbafe71140bc20fbfb57ef WatchSource:0}: Error finding container 233a568f01da311e7b08fb0408ec6f79acff9e913acbafe71140bc20fbfb57ef: Status 404 returned error can't find the container with id 233a568f01da311e7b08fb0408ec6f79acff9e913acbafe71140bc20fbfb57ef
Jul 06 15:27:27.292912 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:27.292880    2580 generic.go:356] "Generic (PLEG): container finished" podID="17725c45-8189-4c10-a419-29f80808d86a" containerID="732ed9ba39717e38b7a0aea9c01110bbbe3d3bc86dd9bdd0bb7384bb79270992" exitCode=0
Jul 06 15:27:27.293297 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:27.292964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerDied","Data":"732ed9ba39717e38b7a0aea9c01110bbbe3d3bc86dd9bdd0bb7384bb79270992"}
Jul 06 15:27:27.294721 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:27.294664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5t8j5" event={"ID":"7b625305-ec07-489b-9d17-6b0a5262a856","Type":"ContainerStarted","Data":"233a568f01da311e7b08fb0408ec6f79acff9e913acbafe71140bc20fbfb57ef"}
Jul 06 15:27:28.281524 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:28.281313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:28.281745 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:28.281484    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:28.281745 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:28.281643    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:27:32.2816252 +0000 UTC m=+44.094487957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:27:28.301459 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:28.301427    2580 generic.go:356] "Generic (PLEG): container finished" podID="17725c45-8189-4c10-a419-29f80808d86a" containerID="743c87d1d584e16bfb8728fc98d1cb96dfb0e6ded496d5c378c1f22274522f83" exitCode=0
Jul 06 15:27:28.301898 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:28.301498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerDied","Data":"743c87d1d584e16bfb8728fc98d1cb96dfb0e6ded496d5c378c1f22274522f83"}
Jul 06 15:27:29.308631 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:29.308586    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" event={"ID":"17725c45-8189-4c10-a419-29f80808d86a","Type":"ContainerStarted","Data":"972f286a2c3c4128dbcfa981afda5ffe748e049badeef1577d7fa903c1324411"}
Jul 06 15:27:29.310257 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:29.310181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5t8j5" event={"ID":"7b625305-ec07-489b-9d17-6b0a5262a856","Type":"ContainerStarted","Data":"7f2f9e84c7e0d1c4dc7ed7bdff84bf31beb0e8be6f23e52eaedcfb8eb3adb7c0"}
Jul 06 15:27:29.336952 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:29.336890    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wvdqh" podStartSLOduration=10.003832886 podStartE2EDuration="40.336872276s" totalImagesPullingTime="30.33303939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:29.336057809 +0000 UTC m=+41.148920599" watchObservedRunningTime="2026-07-06 15:27:29.336872276 +0000 UTC m=+41.149735054"
Jul 06 15:27:29.351482 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:29.351417    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5t8j5" podStartSLOduration=2.650800197 podStartE2EDuration="5.351397228s" totalImagesPullingTime="2.700597031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:27:29.351067636 +0000 UTC m=+41.163930424" watchObservedRunningTime="2026-07-06 15:27:29.351397228 +0000 UTC m=+41.164260008"
Jul 06 15:27:32.314522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:32.314480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:32.314970 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:32.314647    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:32.314970 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:32.314739    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:27:40.314721383 +0000 UTC m=+52.127584142 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:27:40.374598 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:40.374560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:40.375118 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:40.374732    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:40.375118 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:40.374802    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:27:56.37477811 +0000 UTC m=+68.187640865 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:27:50.291808 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:50.291777    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-52rsz"
Jul 06 15:27:54.373303 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.373262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:27:54.376019 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.375997    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 15:27:54.384193 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:54.384162    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 15:27:54.384270 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:54.384260    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs podName:9fd83a4d-6fff-4447-a374-13e06a499206 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:58.384241018 +0000 UTC m=+130.197103775 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs") pod "network-metrics-daemon-rlzvq" (UID: "9fd83a4d-6fff-4447-a374-13e06a499206") : secret "metrics-daemon-secret" not found
Jul 06 15:27:54.474380 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.474329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:54.476995 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.476976    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 15:27:54.488453 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.488431    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/453f039d-a9cd-468d-88d5-34edaa9d8ef0-original-pull-secret\") pod \"global-pull-secret-syncer-5d7zj\" (UID: \"453f039d-a9cd-468d-88d5-34edaa9d8ef0\") " pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:54.575240 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.575187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:54.578096 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.578070    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 15:27:54.589467 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.589441    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 15:27:54.598788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.598754    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2694d\" (UniqueName: \"kubernetes.io/projected/dafdb3cb-d443-4b56-a972-f9c73edae23d-kube-api-access-2694d\") pod \"network-check-target-ghq4d\" (UID: \"dafdb3cb-d443-4b56-a972-f9c73edae23d\") " pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:54.733147 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.733060    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5xpkj\""
Jul 06 15:27:54.740227 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.740200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:27:54.741468 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.741443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5d7zj"
Jul 06 15:27:54.886981 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.886922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5d7zj"]
Jul 06 15:27:54.901756 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:54.901721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ghq4d"]
Jul 06 15:27:54.905381 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:27:54.905349    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddafdb3cb_d443_4b56_a972_f9c73edae23d.slice/crio-250d6853a0702cf215cc02b71d27c5490cc2cac2e41695e42b7471e8c84fcca9 WatchSource:0}: Error finding container 250d6853a0702cf215cc02b71d27c5490cc2cac2e41695e42b7471e8c84fcca9: Status 404 returned error can't find the container with id 250d6853a0702cf215cc02b71d27c5490cc2cac2e41695e42b7471e8c84fcca9
Jul 06 15:27:55.374319 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:55.374281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ghq4d" event={"ID":"dafdb3cb-d443-4b56-a972-f9c73edae23d","Type":"ContainerStarted","Data":"250d6853a0702cf215cc02b71d27c5490cc2cac2e41695e42b7471e8c84fcca9"}
Jul 06 15:27:55.375522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:55.375497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5d7zj" event={"ID":"453f039d-a9cd-468d-88d5-34edaa9d8ef0","Type":"ContainerStarted","Data":"92ada801f9fd27105b3a50e6c0822c32d8f1449e727ef07fa238471d48b99141"}
Jul 06 15:27:56.387461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:27:56.387423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:27:56.387916 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:56.387600    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 15:27:56.387916 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:27:56.387670    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls podName:328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e nodeName:}" failed. No retries permitted until 2026-07-06 15:28:28.387651539 +0000 UTC m=+100.200514296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls") pod "dns-default-r8n47" (UID: "328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e") : secret "dns-default-metrics-tls" not found
Jul 06 15:28:00.391817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:00.391775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5d7zj" event={"ID":"453f039d-a9cd-468d-88d5-34edaa9d8ef0","Type":"ContainerStarted","Data":"9e9b2b535f7d49d3b9e88facd8a5fc119cd729235f79c23115d0c24e66ddb3c3"}
Jul 06 15:28:00.393276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:00.393249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ghq4d" event={"ID":"dafdb3cb-d443-4b56-a972-f9c73edae23d","Type":"ContainerStarted","Data":"93824af04b8f7808e724f9892317730a8b0c46e77d379c2fdcfbcf86b937f2b1"}
Jul 06 15:28:00.393397 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:00.393380    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:28:00.409287 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:00.409220    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5d7zj" podStartSLOduration=65.553847526 podStartE2EDuration="1m10.409202171s" totalImagesPullingTime="4.855354645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:00.409014672 +0000 UTC m=+72.221877450" watchObservedRunningTime="2026-07-06 15:28:00.409202171 +0000 UTC m=+72.222064946"
Jul 06 15:28:00.424061 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:00.424012    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ghq4d" podStartSLOduration=65.596224504 podStartE2EDuration="1m10.423994268s" totalImagesPullingTime="4.827769764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:00.423133068 +0000 UTC m=+72.235995846" watchObservedRunningTime="2026-07-06 15:28:00.423994268 +0000 UTC m=+72.236857095"
Jul 06 15:28:05.023212 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.023179    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"]
Jul 06 15:28:05.025353 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.025325    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.026905 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.026881    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7468768f6c-cbztv"]
Jul 06 15:28:05.027443 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.027423    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 15:28:05.027538 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.027526    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 06 15:28:05.027605 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.027528    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 15:28:05.027726 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.027690    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-v87tq\""
Jul 06 15:28:05.029109 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.029086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.032875 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.032853    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 06 15:28:05.033897 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.033878    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 15:28:05.034038 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.034018    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 15:28:05.034116 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.034075    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 06 15:28:05.034184 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.034117    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bcz88\""
Jul 06 15:28:05.034184 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.034122    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 06 15:28:05.034184 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.034132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 06 15:28:05.034969 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.034732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 06 15:28:05.035992 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.035972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"]
Jul 06 15:28:05.044560 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.044535    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7468768f6c-cbztv"]
Jul 06 15:28:05.054587 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.054540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.054924 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.054907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dvzt\" (UniqueName: \"kubernetes.io/projected/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-kube-api-access-2dvzt\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.055142 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.055123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.123597 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.123558    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"]
Jul 06 15:28:05.125792 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.125771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"
Jul 06 15:28:05.129728 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.129710    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jvr2r\""
Jul 06 15:28:05.129728 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.129717    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 06 15:28:05.131220 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.131196    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-646b498d44-b5d4g"]
Jul 06 15:28:05.133763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.133743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.135139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.135121    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 06 15:28:05.136983 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.136964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 15:28:05.137577 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.137560    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"]
Jul 06 15:28:05.141111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.141091    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 15:28:05.141270 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.141128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pngzd\""
Jul 06 15:28:05.141644 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.141620    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 15:28:05.142904 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.142884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 15:28:05.148780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.148742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-646b498d44-b5d4g"]
Jul 06 15:28:05.155523 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dvzt\" (UniqueName: \"kubernetes.io/projected/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-kube-api-access-2dvzt\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.155693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-stats-auth\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.155693 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.155826 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.155826 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.155826 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb547\" (UniqueName: \"kubernetes.io/projected/42d7fee8-aac1-4d87-9034-8632800488f7-kube-api-access-hb547\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.155826 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-default-certificate\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.155826 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.155799    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:05.156105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.156105 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.155919    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls podName:6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:05.655894361 +0000 UTC m=+77.468757127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x8d7q" (UID: "6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9") : secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:05.156105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.155989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v9gp\" (UniqueName: \"kubernetes.io/projected/c0d4dba4-08b8-45c2-986d-15c6bd46954f-kube-api-access-4v9gp\") pod \"volume-data-source-validator-5d456fdbcd-dd24v\" (UID: \"c0d4dba4-08b8-45c2-986d-15c6bd46954f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"
Jul 06 15:28:05.156540 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.156521    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.173682 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.173642    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dvzt\" (UniqueName: \"kubernetes.io/projected/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-kube-api-access-2dvzt\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.256856 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.256815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-stats-auth\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.257032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.256879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-image-registry-private-configuration\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.256957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.257032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.256988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb547\" (UniqueName: \"kubernetes.io/projected/42d7fee8-aac1-4d87-9034-8632800488f7-kube-api-access-hb547\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.257032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-certificates\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-default-certificate\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d200281-ce2e-445e-b6d2-bdf380db6d25-ca-trust-extracted\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.257149    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:05.75712311 +0000 UTC m=+77.569985883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : configmap references non-existent config key: service-ca.crt
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v9gp\" (UniqueName: \"kubernetes.io/projected/c0d4dba4-08b8-45c2-986d-15c6bd46954f-kube-api-access-4v9gp\") pod \"volume-data-source-validator-5d456fdbcd-dd24v\" (UID: \"c0d4dba4-08b8-45c2-986d-15c6bd46954f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.257200    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 15:28:05.257234 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-bound-sa-token\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257522 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.257249    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:05.757232328 +0000 UTC m=+77.570095084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : secret "router-metrics-certs-default" not found
Jul 06 15:28:05.257522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqg9z\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-kube-api-access-lqg9z\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-trusted-ca\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.257522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.257414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-installation-pull-secrets\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.259422 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.259403    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-stats-auth\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.259479 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.259443    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-default-certificate\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.268857 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.268818    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb547\" (UniqueName: \"kubernetes.io/projected/42d7fee8-aac1-4d87-9034-8632800488f7-kube-api-access-hb547\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.272294 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.272269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v9gp\" (UniqueName: \"kubernetes.io/projected/c0d4dba4-08b8-45c2-986d-15c6bd46954f-kube-api-access-4v9gp\") pod \"volume-data-source-validator-5d456fdbcd-dd24v\" (UID: \"c0d4dba4-08b8-45c2-986d-15c6bd46954f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"
Jul 06 15:28:05.358624 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-trusted-ca\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358624 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358624 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-installation-pull-secrets\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-image-registry-private-configuration\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-certificates\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d200281-ce2e-445e-b6d2-bdf380db6d25-ca-trust-extracted\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-bound-sa-token\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.358790    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.358803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqg9z\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-kube-api-access-lqg9z\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.358813    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646b498d44-b5d4g: secret "image-registry-tls" not found
Jul 06 15:28:05.358946 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.358904    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls podName:1d200281-ce2e-445e-b6d2-bdf380db6d25 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:05.858883592 +0000 UTC m=+77.671746369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls") pod "image-registry-646b498d44-b5d4g" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25") : secret "image-registry-tls" not found
Jul 06 15:28:05.359341 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.359253    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d200281-ce2e-445e-b6d2-bdf380db6d25-ca-trust-extracted\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.359452 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.359431    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-certificates\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.359678 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.359655    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-trusted-ca\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.361684 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.361665    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-installation-pull-secrets\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.361752 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.361720    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-image-registry-private-configuration\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.370194 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.370163    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqg9z\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-kube-api-access-lqg9z\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.370445 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.370426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-bound-sa-token\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.435834 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.435791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"
Jul 06 15:28:05.564655 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.564617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v"]
Jul 06 15:28:05.568957 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:05.568931    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d4dba4_08b8_45c2_986d_15c6bd46954f.slice/crio-2aba11aaa85e0e668abb35bb1a38a56da4c307936daae3deac930a64fb2a1a46 WatchSource:0}: Error finding container 2aba11aaa85e0e668abb35bb1a38a56da4c307936daae3deac930a64fb2a1a46: Status 404 returned error can't find the container with id 2aba11aaa85e0e668abb35bb1a38a56da4c307936daae3deac930a64fb2a1a46
Jul 06 15:28:05.660782 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.660746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:05.660982 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.660912    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:05.660982 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.660976    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls podName:6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:06.66096036 +0000 UTC m=+78.473823116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x8d7q" (UID: "6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9") : secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:05.761646 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.761599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.761866 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.761738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:05.761866 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.761788    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 15:28:05.761866 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.761863    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:06.761845776 +0000 UTC m=+78.574708534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : configmap references non-existent config key: service-ca.crt
Jul 06 15:28:05.762002 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.761880    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:06.761873405 +0000 UTC m=+78.574736161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : secret "router-metrics-certs-default" not found
Jul 06 15:28:05.862320 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:05.862283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:05.862472 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.862438    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 15:28:05.862472 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.862458    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646b498d44-b5d4g: secret "image-registry-tls" not found
Jul 06 15:28:05.862543 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:05.862520    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls podName:1d200281-ce2e-445e-b6d2-bdf380db6d25 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:06.862496506 +0000 UTC m=+78.675359262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls") pod "image-registry-646b498d44-b5d4g" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25") : secret "image-registry-tls" not found
Jul 06 15:28:06.409029 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:06.408988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v" event={"ID":"c0d4dba4-08b8-45c2-986d-15c6bd46954f","Type":"ContainerStarted","Data":"2aba11aaa85e0e668abb35bb1a38a56da4c307936daae3deac930a64fb2a1a46"}
Jul 06 15:28:06.669153 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:06.669053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:06.669301 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.669216    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:06.669301 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.669286    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls podName:6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:08.669268342 +0000 UTC m=+80.482131098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x8d7q" (UID: "6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9") : secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:06.770072 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:06.770038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:06.770238 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:06.770084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:06.770238 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.770178    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 15:28:06.770238 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.770209    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:08.77019176 +0000 UTC m=+80.583054516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : configmap references non-existent config key: service-ca.crt
Jul 06 15:28:06.770238 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.770232    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:08.770224936 +0000 UTC m=+80.583087691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : secret "router-metrics-certs-default" not found
Jul 06 15:28:06.870549 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:06.870515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:06.870725 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.870670    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 15:28:06.870725 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.870690    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646b498d44-b5d4g: secret "image-registry-tls" not found
Jul 06 15:28:06.870802 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:06.870773    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls podName:1d200281-ce2e-445e-b6d2-bdf380db6d25 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:08.870750259 +0000 UTC m=+80.683613020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls") pod "image-registry-646b498d44-b5d4g" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25") : secret "image-registry-tls" not found
Jul 06 15:28:07.413324 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:07.413261    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v" event={"ID":"c0d4dba4-08b8-45c2-986d-15c6bd46954f","Type":"ContainerStarted","Data":"3bd014b88319a399db3ec0d67ae177ace724ca88eec75e0208f9c270d25857f1"}
Jul 06 15:28:07.431309 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:07.431255    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dd24v" podStartSLOduration=1.477737951 podStartE2EDuration="2.431238536s" totalImagesPullingTime="953.500585ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:07.43000615 +0000 UTC m=+79.242868929" watchObservedRunningTime="2026-07-06 15:28:07.431238536 +0000 UTC m=+79.244101353"
Jul 06 15:28:08.684927 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:08.684873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:08.685342 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.685019    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:08.685342 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.685092    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls podName:6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:12.685075878 +0000 UTC m=+84.497938635 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x8d7q" (UID: "6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9") : secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:08.785641 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:08.785605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:08.785823 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:08.785694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:08.785823 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.785783    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 15:28:08.785898 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.785822    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:12.785804034 +0000 UTC m=+84.598666789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : configmap references non-existent config key: service-ca.crt
Jul 06 15:28:08.785898 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.785843    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:12.785836233 +0000 UTC m=+84.598698990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : secret "router-metrics-certs-default" not found
Jul 06 15:28:08.886266 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:08.886239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:08.886408 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.886389    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 15:28:08.886443 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.886411    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646b498d44-b5d4g: secret "image-registry-tls" not found
Jul 06 15:28:08.886473 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:08.886466    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls podName:1d200281-ce2e-445e-b6d2-bdf380db6d25 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:12.88645024 +0000 UTC m=+84.699312996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls") pod "image-registry-646b498d44-b5d4g" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25") : secret "image-registry-tls" not found
Jul 06 15:28:10.942989 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:10.942865    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"]
Jul 06 15:28:10.944742 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:10.944727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"
Jul 06 15:28:10.946968 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:10.946948    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 15:28:10.947559 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:10.947541    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 15:28:10.947617 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:10.947588    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r7dx2\""
Jul 06 15:28:10.961250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:10.961219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"]
Jul 06 15:28:11.005038 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.004995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwmmf\" (UniqueName: \"kubernetes.io/projected/22e30d44-1639-4f52-bc79-cea5427f82fd-kube-api-access-hwmmf\") pod \"migrator-d57d58cc9-jblbm\" (UID: \"22e30d44-1639-4f52-bc79-cea5427f82fd\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"
Jul 06 15:28:11.105849 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.105817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwmmf\" (UniqueName: \"kubernetes.io/projected/22e30d44-1639-4f52-bc79-cea5427f82fd-kube-api-access-hwmmf\") pod \"migrator-d57d58cc9-jblbm\" (UID: \"22e30d44-1639-4f52-bc79-cea5427f82fd\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"
Jul 06 15:28:11.123268 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.123234    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwmmf\" (UniqueName: \"kubernetes.io/projected/22e30d44-1639-4f52-bc79-cea5427f82fd-kube-api-access-hwmmf\") pod \"migrator-d57d58cc9-jblbm\" (UID: \"22e30d44-1639-4f52-bc79-cea5427f82fd\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"
Jul 06 15:28:11.144917 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.144883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g5ccn"]
Jul 06 15:28:11.147078 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.147063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.150894 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.150871    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7qvb\""
Jul 06 15:28:11.151017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.150916    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 15:28:11.151611 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.151594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 15:28:11.151736 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.151721    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 15:28:11.151776 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.151721    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 15:28:11.163678 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.163647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g5ccn"]
Jul 06 15:28:11.206772 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.206660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58ed0bdd-c406-4dbe-8659-5595955f68ff-crio-socket\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.206772 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.206727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.206772 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.206757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hm2f\" (UniqueName: \"kubernetes.io/projected/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-api-access-6hm2f\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.206979 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.206883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58ed0bdd-c406-4dbe-8659-5595955f68ff-data-volume\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.206979 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.206918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.253406 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.253371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"
Jul 06 15:28:11.307838 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.307802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58ed0bdd-c406-4dbe-8659-5595955f68ff-crio-socket\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.308032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.307863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.308032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.307894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hm2f\" (UniqueName: \"kubernetes.io/projected/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-api-access-6hm2f\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.308032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.307924    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/58ed0bdd-c406-4dbe-8659-5595955f68ff-crio-socket\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.308032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.308000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58ed0bdd-c406-4dbe-8659-5595955f68ff-data-volume\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.308032 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.308004    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 15:28:11.308387 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.308039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.308387 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.308076    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:11.808056845 +0000 UTC m=+83.620919606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : configmap "kube-rbac-proxy" not found
Jul 06 15:28:11.308387 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.308107    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:11.308387 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.308150    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:11.80813549 +0000 UTC m=+83.620998252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:11.308387 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.308353    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/58ed0bdd-c406-4dbe-8659-5595955f68ff-data-volume\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.320620 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.320589    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hm2f\" (UniqueName: \"kubernetes.io/projected/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-api-access-6hm2f\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.388746 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.388713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm"]
Jul 06 15:28:11.391894 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:11.391864    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e30d44_1639_4f52_bc79_cea5427f82fd.slice/crio-c05c30790a0ffc60302b00ed1fb4bd9fe5c0033383ff44dafdde324b2cec704d WatchSource:0}: Error finding container c05c30790a0ffc60302b00ed1fb4bd9fe5c0033383ff44dafdde324b2cec704d: Status 404 returned error can't find the container with id c05c30790a0ffc60302b00ed1fb4bd9fe5c0033383ff44dafdde324b2cec704d
Jul 06 15:28:11.428794 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.428751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm" event={"ID":"22e30d44-1639-4f52-bc79-cea5427f82fd","Type":"ContainerStarted","Data":"c05c30790a0ffc60302b00ed1fb4bd9fe5c0033383ff44dafdde324b2cec704d"}
Jul 06 15:28:11.812454 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.812411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.812663 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:11.812499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:11.812663 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.812567    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 15:28:11.812663 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.812608    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:11.812663 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.812650    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:12.812628573 +0000 UTC m=+84.625491334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : configmap "kube-rbac-proxy" not found
Jul 06 15:28:11.812927 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:11.812670    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:12.812661059 +0000 UTC m=+84.625523815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:12.721692 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:12.721659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:12.722055 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.721839    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:12.722055 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.721929    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls podName:6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:20.721904904 +0000 UTC m=+92.534767678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x8d7q" (UID: "6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9") : secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:12.823126 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:12.823089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:12.823126 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:12.823131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:12.823347 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:12.823153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:12.823347 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:12.823216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:12.823347 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823261    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 15:28:12.823347 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823284    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 15:28:12.823347 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823266    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:20.823243536 +0000 UTC m=+92.636106295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : configmap references non-existent config key: service-ca.crt
Jul 06 15:28:12.823347 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823328    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:12.823534 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823335    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:20.823318515 +0000 UTC m=+92.636181275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : secret "router-metrics-certs-default" not found
Jul 06 15:28:12.823534 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823382    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:14.82337012 +0000 UTC m=+86.636232879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : configmap "kube-rbac-proxy" not found
Jul 06 15:28:12.823534 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.823393    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:14.823387087 +0000 UTC m=+86.636249843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:12.924512 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:12.924462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:12.924732 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.924623    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 15:28:12.924732 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.924642    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-646b498d44-b5d4g: secret "image-registry-tls" not found
Jul 06 15:28:12.924732 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:12.924726    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls podName:1d200281-ce2e-445e-b6d2-bdf380db6d25 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:20.924687509 +0000 UTC m=+92.737550266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls") pod "image-registry-646b498d44-b5d4g" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25") : secret "image-registry-tls" not found
Jul 06 15:28:13.436989 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:13.436947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm" event={"ID":"22e30d44-1639-4f52-bc79-cea5427f82fd","Type":"ContainerStarted","Data":"12390de4e367b8cbb7f258a6b4fd44b0cb273e57ce8804962445b4391ab09121"}
Jul 06 15:28:13.437167 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:13.437015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm" event={"ID":"22e30d44-1639-4f52-bc79-cea5427f82fd","Type":"ContainerStarted","Data":"1119961f8f0b355755b786c4b2561c49cda1bf8415d75eca2452305696e26353"}
Jul 06 15:28:13.462247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:13.462195    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-jblbm" podStartSLOduration=2.165548728 podStartE2EDuration="3.462180221s" totalImagesPullingTime="1.296631493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:13.461678327 +0000 UTC m=+85.274541104" watchObservedRunningTime="2026-07-06 15:28:13.462180221 +0000 UTC m=+85.275042998"
Jul 06 15:28:13.921814 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:13.921783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lf2hj_4af80c0b-5938-4740-8c22-dfac3b02e1d0/dns-node-resolver/0.log"
Jul 06 15:28:14.719812 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:14.719783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fkmm6_0c2c1f01-a050-4040-a6c8-b39fd086a324/node-ca/0.log"
Jul 06 15:28:14.841445 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:14.841407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:14.841546 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:14.841529    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 15:28:14.841605 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:14.841595    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:18.8415802 +0000 UTC m=+90.654442955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : configmap "kube-rbac-proxy" not found
Jul 06 15:28:14.841665 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:14.841588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:14.841742 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:14.841662    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:14.841742 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:14.841730    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:18.841695265 +0000 UTC m=+90.654558021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:15.723688 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:15.723651    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5t8j5_7b625305-ec07-489b-9d17-6b0a5262a856/serve-healthcheck-canary/0.log"
Jul 06 15:28:16.119972 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:16.119943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-jblbm_22e30d44-1639-4f52-bc79-cea5427f82fd/migrator/0.log"
Jul 06 15:28:16.320636 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:16.320597    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-jblbm_22e30d44-1639-4f52-bc79-cea5427f82fd/graceful-termination/0.log"
Jul 06 15:28:18.878976 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:18.878934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:18.879336 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:18.879077    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 15:28:18.879336 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:18.879117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:18.879336 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:18.879167    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:26.879146856 +0000 UTC m=+98.692009613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : configmap "kube-rbac-proxy" not found
Jul 06 15:28:18.879336 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:18.879225    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:18.879336 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:18.879261    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls podName:58ed0bdd-c406-4dbe-8659-5595955f68ff nodeName:}" failed. No retries permitted until 2026-07-06 15:28:26.879252693 +0000 UTC m=+98.692115453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g5ccn" (UID: "58ed0bdd-c406-4dbe-8659-5595955f68ff") : secret "insights-runtime-extractor-tls" not found
Jul 06 15:28:20.795262 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:20.795202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:20.795720 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:20.795345    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:20.795720 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:20.795412    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls podName:6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:36.795395736 +0000 UTC m=+108.608258493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x8d7q" (UID: "6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9") : secret "cluster-monitoring-operator-tls" not found
Jul 06 15:28:20.896352 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:20.896309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:20.896352 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:20.896352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:20.896587 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:20.896565    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle podName:42d7fee8-aac1-4d87-9034-8632800488f7 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:36.896546771 +0000 UTC m=+108.709409539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle") pod "router-default-7468768f6c-cbztv" (UID: "42d7fee8-aac1-4d87-9034-8632800488f7") : configmap references non-existent config key: service-ca.crt
Jul 06 15:28:20.898662 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:20.898631    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42d7fee8-aac1-4d87-9034-8632800488f7-metrics-certs\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:20.997606 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:20.997549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:21.000094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:21.000071    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"image-registry-646b498d44-b5d4g\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") " pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:21.043125 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:21.043088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:21.195269 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:21.195238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-646b498d44-b5d4g"]
Jul 06 15:28:21.198945 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:21.198903    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d200281_ce2e_445e_b6d2_bdf380db6d25.slice/crio-040f4556b39082c25e8ed47318417f9587e41708daa9c96d8a3f97387eec40cc WatchSource:0}: Error finding container 040f4556b39082c25e8ed47318417f9587e41708daa9c96d8a3f97387eec40cc: Status 404 returned error can't find the container with id 040f4556b39082c25e8ed47318417f9587e41708daa9c96d8a3f97387eec40cc
Jul 06 15:28:21.465414 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:21.465376    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" event={"ID":"1d200281-ce2e-445e-b6d2-bdf380db6d25","Type":"ContainerStarted","Data":"4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd"}
Jul 06 15:28:21.465571 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:21.465432    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" event={"ID":"1d200281-ce2e-445e-b6d2-bdf380db6d25","Type":"ContainerStarted","Data":"040f4556b39082c25e8ed47318417f9587e41708daa9c96d8a3f97387eec40cc"}
Jul 06 15:28:21.465571 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:21.465517    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:26.946912 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:26.946871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:26.947301 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:26.946936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:26.947411 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:26.947394    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/58ed0bdd-c406-4dbe-8659-5595955f68ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:26.949191 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:26.949170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/58ed0bdd-c406-4dbe-8659-5595955f68ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g5ccn\" (UID: \"58ed0bdd-c406-4dbe-8659-5595955f68ff\") " pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:27.056719 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:27.056669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g5ccn"
Jul 06 15:28:27.188182 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:27.188081    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" podStartSLOduration=22.188056317 podStartE2EDuration="22.188056317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:21.495291994 +0000 UTC m=+93.308154772" watchObservedRunningTime="2026-07-06 15:28:27.188056317 +0000 UTC m=+99.000919097"
Jul 06 15:28:27.188391 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:27.188374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g5ccn"]
Jul 06 15:28:27.192054 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:27.192010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ed0bdd_c406_4dbe_8659_5595955f68ff.slice/crio-f3ea22dd90769fe57a97d6928f82c9388cfcb901860df274d005ee199c0a09e3 WatchSource:0}: Error finding container f3ea22dd90769fe57a97d6928f82c9388cfcb901860df274d005ee199c0a09e3: Status 404 returned error can't find the container with id f3ea22dd90769fe57a97d6928f82c9388cfcb901860df274d005ee199c0a09e3
Jul 06 15:28:27.489462 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:27.489428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5ccn" event={"ID":"58ed0bdd-c406-4dbe-8659-5595955f68ff","Type":"ContainerStarted","Data":"f3ea22dd90769fe57a97d6928f82c9388cfcb901860df274d005ee199c0a09e3"}
Jul 06 15:28:28.459823 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:28.459785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:28:28.462529 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:28.462506    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e-metrics-tls\") pod \"dns-default-r8n47\" (UID: \"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e\") " pod="openshift-dns/dns-default-r8n47"
Jul 06 15:28:28.467907 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:28.467871    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cwwg8\""
Jul 06 15:28:28.475904 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:28.475875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8n47"
Jul 06 15:28:28.626615 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:28.626585    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8n47"]
Jul 06 15:28:28.630658 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:28.630624    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328c2ae1_5a8d_49b2_84b0_7ff8611a9e1e.slice/crio-788048d175e32a2ae5ad26718e4eb5610179acaf4d15a305931087ccb06fa352 WatchSource:0}: Error finding container 788048d175e32a2ae5ad26718e4eb5610179acaf4d15a305931087ccb06fa352: Status 404 returned error can't find the container with id 788048d175e32a2ae5ad26718e4eb5610179acaf4d15a305931087ccb06fa352
Jul 06 15:28:29.498922 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:29.498887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8n47" event={"ID":"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e","Type":"ContainerStarted","Data":"788048d175e32a2ae5ad26718e4eb5610179acaf4d15a305931087ccb06fa352"}
Jul 06 15:28:31.399294 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:31.399258    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ghq4d"
Jul 06 15:28:32.512807 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:32.512768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5ccn" event={"ID":"58ed0bdd-c406-4dbe-8659-5595955f68ff","Type":"ContainerStarted","Data":"f53e3fe3541404392342eb5f6fcbdff1e3d400d7cb7a1e65b77799e6b04b9672"}
Jul 06 15:28:32.514365 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:32.514344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8n47" event={"ID":"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e","Type":"ContainerStarted","Data":"ec6b7c20769a4f5e314babc518138c0f1975c51d7de501ae7eb8a03a9d5afcc1"}
Jul 06 15:28:32.514472 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:32.514382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8n47" event={"ID":"328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e","Type":"ContainerStarted","Data":"36f18946438769a620545a537d9bc5bec4d398bcaf19dff5d45f08017e15fdbe"}
Jul 06 15:28:32.514530 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:32.514479    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r8n47"
Jul 06 15:28:32.541087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:32.541035    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r8n47" podStartSLOduration=65.687274622 podStartE2EDuration="1m8.541012689s" totalImagesPullingTime="2.853738067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:32.540533175 +0000 UTC m=+104.353395953" watchObservedRunningTime="2026-07-06 15:28:32.541012689 +0000 UTC m=+104.353875479"
Jul 06 15:28:33.520716 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:33.520652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5ccn" event={"ID":"58ed0bdd-c406-4dbe-8659-5595955f68ff","Type":"ContainerStarted","Data":"9d9f86e868e0c59e732c30d7ab4ade53d45ad106fd5aea6f2533c61591499d7b"}
Jul 06 15:28:35.529751 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:35.529689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g5ccn" event={"ID":"58ed0bdd-c406-4dbe-8659-5595955f68ff","Type":"ContainerStarted","Data":"baf7c63decbd8541f6ec068bd3c1a41ce20f140d2b3488ca2ddc014dd4bd8503"}
Jul 06 15:28:35.553086 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:35.552865    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g5ccn" podStartSLOduration=16.633434245 podStartE2EDuration="24.552847515s" totalImagesPullingTime="7.91941327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 15:28:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:35.552591037 +0000 UTC m=+107.365453827" watchObservedRunningTime="2026-07-06 15:28:35.552847515 +0000 UTC m=+107.365710294"
Jul 06 15:28:36.248672 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.248633    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"]
Jul 06 15:28:36.250839 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.250821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.254016 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.253984    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 15:28:36.254147 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.253984    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 15:28:36.254147 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.254019    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rsnjw\""
Jul 06 15:28:36.260393 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.260366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"]
Jul 06 15:28:36.418019 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.417983    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-646b498d44-b5d4g"]
Jul 06 15:28:36.422424 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.422383    2580 patch_prober.go:28] interesting pod/image-registry-646b498d44-b5d4g container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 15:28:36.422582 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.422439    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 15:28:36.430824 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.430793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaca221-afba-4397-a744-e5ef87e1f5df-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-hdj6t\" (UID: \"baaca221-afba-4397-a744-e5ef87e1f5df\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.430990 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.430830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/baaca221-afba-4397-a744-e5ef87e1f5df-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-hdj6t\" (UID: \"baaca221-afba-4397-a744-e5ef87e1f5df\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.531398 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.531290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaca221-afba-4397-a744-e5ef87e1f5df-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-hdj6t\" (UID: \"baaca221-afba-4397-a744-e5ef87e1f5df\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.531398 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.531336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/baaca221-afba-4397-a744-e5ef87e1f5df-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-hdj6t\" (UID: \"baaca221-afba-4397-a744-e5ef87e1f5df\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.532221 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.532199    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/baaca221-afba-4397-a744-e5ef87e1f5df-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-hdj6t\" (UID: \"baaca221-afba-4397-a744-e5ef87e1f5df\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.534379 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.534331    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaca221-afba-4397-a744-e5ef87e1f5df-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-hdj6t\" (UID: \"baaca221-afba-4397-a744-e5ef87e1f5df\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.559583 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.559533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"
Jul 06 15:28:36.692137 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.692104    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t"]
Jul 06 15:28:36.694694 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:36.694659    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaaca221_afba_4397_a744_e5ef87e1f5df.slice/crio-c8b02db9ae6bb5dc30a642f9f8b96731d5b481e45d648407c376ab23773761db WatchSource:0}: Error finding container c8b02db9ae6bb5dc30a642f9f8b96731d5b481e45d648407c376ab23773761db: Status 404 returned error can't find the container with id c8b02db9ae6bb5dc30a642f9f8b96731d5b481e45d648407c376ab23773761db
Jul 06 15:28:36.833571 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.833477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:36.836030 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.836003    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x8d7q\" (UID: \"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:36.838789 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.838762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"
Jul 06 15:28:36.934192 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.934161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:36.934747 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.934729    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d7fee8-aac1-4d87-9034-8632800488f7-service-ca-bundle\") pod \"router-default-7468768f6c-cbztv\" (UID: \"42d7fee8-aac1-4d87-9034-8632800488f7\") " pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:36.975524 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:36.975487    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q"]
Jul 06 15:28:36.979096 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:36.979069    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c04b5fb_1ad5_4bf7_95f3_e56e9eae47c9.slice/crio-e7eeb29e380a46331531c1646bdb8146dcab491e1d2ab70227d6e1cac3804023 WatchSource:0}: Error finding container e7eeb29e380a46331531c1646bdb8146dcab491e1d2ab70227d6e1cac3804023: Status 404 returned error can't find the container with id e7eeb29e380a46331531c1646bdb8146dcab491e1d2ab70227d6e1cac3804023
Jul 06 15:28:37.144605 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.144565    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:37.282118 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.282081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7468768f6c-cbztv"]
Jul 06 15:28:37.285807 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:37.285769    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d7fee8_aac1_4d87_9034_8632800488f7.slice/crio-cfcbfd143aef4ff74037ca6f65a87afd95368fd8bc79d4ddd3d3ebb3e788401c WatchSource:0}: Error finding container cfcbfd143aef4ff74037ca6f65a87afd95368fd8bc79d4ddd3d3ebb3e788401c: Status 404 returned error can't find the container with id cfcbfd143aef4ff74037ca6f65a87afd95368fd8bc79d4ddd3d3ebb3e788401c
Jul 06 15:28:37.540264 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.540137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q" event={"ID":"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9","Type":"ContainerStarted","Data":"e7eeb29e380a46331531c1646bdb8146dcab491e1d2ab70227d6e1cac3804023"}
Jul 06 15:28:37.541511 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.541485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t" event={"ID":"baaca221-afba-4397-a744-e5ef87e1f5df","Type":"ContainerStarted","Data":"c8b02db9ae6bb5dc30a642f9f8b96731d5b481e45d648407c376ab23773761db"}
Jul 06 15:28:37.543105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.543061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7468768f6c-cbztv" event={"ID":"42d7fee8-aac1-4d87-9034-8632800488f7","Type":"ContainerStarted","Data":"473ef9650738631f0c4cf392c8d3dcc7114fab9457e998871a644617c0f42d3c"}
Jul 06 15:28:37.543105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.543100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7468768f6c-cbztv" event={"ID":"42d7fee8-aac1-4d87-9034-8632800488f7","Type":"ContainerStarted","Data":"cfcbfd143aef4ff74037ca6f65a87afd95368fd8bc79d4ddd3d3ebb3e788401c"}
Jul 06 15:28:37.566584 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:37.566525    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7468768f6c-cbztv" podStartSLOduration=32.566505326 podStartE2EDuration="32.566505326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:37.565345843 +0000 UTC m=+109.378208614" watchObservedRunningTime="2026-07-06 15:28:37.566505326 +0000 UTC m=+109.379368082"
Jul 06 15:28:38.145262 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:38.145225    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:38.149576 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:38.149545    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:38.549015 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:38.548977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t" event={"ID":"baaca221-afba-4397-a744-e5ef87e1f5df","Type":"ContainerStarted","Data":"4a55fe172add6d150b07c38368e02fa600f2e171d71a7bb9d4e8e6b8ad537079"}
Jul 06 15:28:38.549501 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:38.549166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:38.550715 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:38.550677    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7468768f6c-cbztv"
Jul 06 15:28:38.569259 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:38.569196    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-hdj6t" podStartSLOduration=1.3627955 podStartE2EDuration="2.569177203s" totalImagesPullingTime="1.206381703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:38.568191221 +0000 UTC m=+110.381054000" watchObservedRunningTime="2026-07-06 15:28:38.569177203 +0000 UTC m=+110.382039981"
Jul 06 15:28:39.348342 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.348311    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"]
Jul 06 15:28:39.350620 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.350595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:39.350887 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.350859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49d68ccf-ff59-4f36-9014-d1cb964ac8fb-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kvp6q\" (UID: \"49d68ccf-ff59-4f36-9014-d1cb964ac8fb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:39.352677 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.352653    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 06 15:28:39.352789 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.352730    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rclvv\""
Jul 06 15:28:39.365779 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.365748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"]
Jul 06 15:28:39.451317 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.451279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49d68ccf-ff59-4f36-9014-d1cb964ac8fb-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kvp6q\" (UID: \"49d68ccf-ff59-4f36-9014-d1cb964ac8fb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:39.451499 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:39.451428    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 06 15:28:39.451499 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:39.451492    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49d68ccf-ff59-4f36-9014-d1cb964ac8fb-tls-certificates podName:49d68ccf-ff59-4f36-9014-d1cb964ac8fb nodeName:}" failed. No retries permitted until 2026-07-06 15:28:39.951476288 +0000 UTC m=+111.764339045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/49d68ccf-ff59-4f36-9014-d1cb964ac8fb-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-kvp6q" (UID: "49d68ccf-ff59-4f36-9014-d1cb964ac8fb") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 06 15:28:39.554137 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.554096    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q" event={"ID":"6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9","Type":"ContainerStarted","Data":"8f28b712a88a10227ba90d5dd852ce4a6aa817facf67062452fa0c73bf1a6d6d"}
Jul 06 15:28:39.575512 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.575455    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x8d7q" podStartSLOduration=32.727212246 podStartE2EDuration="34.575440012s" totalImagesPullingTime="1.848227766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:39.574838834 +0000 UTC m=+111.387701615" watchObservedRunningTime="2026-07-06 15:28:39.575440012 +0000 UTC m=+111.388302804"
Jul 06 15:28:39.954994 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.954953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49d68ccf-ff59-4f36-9014-d1cb964ac8fb-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kvp6q\" (UID: \"49d68ccf-ff59-4f36-9014-d1cb964ac8fb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:39.957403 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.957370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49d68ccf-ff59-4f36-9014-d1cb964ac8fb-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-kvp6q\" (UID: \"49d68ccf-ff59-4f36-9014-d1cb964ac8fb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:39.960253 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:39.960229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:40.094679 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:40.094644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"]
Jul 06 15:28:40.098090 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:40.098058    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d68ccf_ff59_4f36_9014_d1cb964ac8fb.slice/crio-4497c00eab70d978ee88936775d7e1a31622ce0aec28831ef095a6068d256b20 WatchSource:0}: Error finding container 4497c00eab70d978ee88936775d7e1a31622ce0aec28831ef095a6068d256b20: Status 404 returned error can't find the container with id 4497c00eab70d978ee88936775d7e1a31622ce0aec28831ef095a6068d256b20
Jul 06 15:28:40.558604 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:40.558557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q" event={"ID":"49d68ccf-ff59-4f36-9014-d1cb964ac8fb","Type":"ContainerStarted","Data":"4497c00eab70d978ee88936775d7e1a31622ce0aec28831ef095a6068d256b20"}
Jul 06 15:28:41.564020 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:41.563932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q" event={"ID":"49d68ccf-ff59-4f36-9014-d1cb964ac8fb","Type":"ContainerStarted","Data":"1b168781e2bd492978a6e0773dec11ec21f81f2a72677409be96ca424c861a07"}
Jul 06 15:28:41.564374 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:41.564093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:41.568724 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:41.568686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q"
Jul 06 15:28:41.581277 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:41.581232    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-kvp6q" podStartSLOduration=1.515421446 podStartE2EDuration="2.581213936s" totalImagesPullingTime="1.06579249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:41.581148044 +0000 UTC m=+113.394010822" watchObservedRunningTime="2026-07-06 15:28:41.581213936 +0000 UTC m=+113.394076716"
Jul 06 15:28:42.522983 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:42.522957    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r8n47"
Jul 06 15:28:46.423465 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.423422    2580 patch_prober.go:28] interesting pod/image-registry-646b498d44-b5d4g container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 15:28:46.423886 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.423502    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 15:28:46.800508 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.800425    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"]
Jul 06 15:28:46.805770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.805743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:46.808299 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.808276    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 15:28:46.808432 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.808395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 06 15:28:46.808507 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.808493    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 06 15:28:46.809593 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.809578    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-p7lvk\""
Jul 06 15:28:46.812447 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.812422    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"]
Jul 06 15:28:46.827881 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.827845    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"]
Jul 06 15:28:46.831775 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.831747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.834315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.834282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 06 15:28:46.834315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.834302    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p84rw\""
Jul 06 15:28:46.834562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.834282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 06 15:28:46.834562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.834412    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 06 15:28:46.834822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.834802    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jtp4m"]
Jul 06 15:28:46.838622 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.838596    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.840382 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.840360    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 15:28:46.840673 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.840657    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 15:28:46.840736 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.840680    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g2nv5\""
Jul 06 15:28:46.840736 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.840659    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 15:28:46.844374 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.844346    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"]
Jul 06 15:28:46.900543 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.900543 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrxn7\" (UniqueName: \"kubernetes.io/projected/fff5e63b-e081-4365-9e55-d7a431d73d23-kube-api-access-lrxn7\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-textfile\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0902663f-3122-4f56-98a7-770cf809d201-metrics-client-ca\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjxnp\" (UniqueName: \"kubernetes.io/projected/0902663f-3122-4f56-98a7-770cf809d201-kube-api-access-wjxnp\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff5e63b-e081-4365-9e55-d7a431d73d23-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj5fc\" (UniqueName: \"kubernetes.io/projected/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-api-access-pj5fc\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.900817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900817    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-accelerators-collector-config\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-root\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-sys\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.900999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.901016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-wtmp\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:46.901093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:46.901033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.002216 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-textfile\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.002416 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.002416 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.002346    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 06 15:28:47.002416 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.002416 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.002410    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls podName:8ed5bba2-e1c8-4c56-86a8-1de04638eafa nodeName:}" failed. No retries permitted until 2026-07-06 15:28:47.502390107 +0000 UTC m=+119.315252863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-v6r42" (UID: "8ed5bba2-e1c8-4c56-86a8-1de04638eafa") : secret "kube-state-metrics-tls" not found
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0902663f-3122-4f56-98a7-770cf809d201-metrics-client-ca\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjxnp\" (UniqueName: \"kubernetes.io/projected/0902663f-3122-4f56-98a7-770cf809d201-kube-api-access-wjxnp\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff5e63b-e081-4365-9e55-d7a431d73d23-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5fc\" (UniqueName: \"kubernetes.io/projected/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-api-access-pj5fc\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-textfile\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.002647 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-accelerators-collector-config\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-root\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-sys\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.002846    2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-wtmp\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.002908    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-tls podName:fff5e63b-e081-4365-9e55-d7a431d73d23 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:47.502892019 +0000 UTC m=+119.315754781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-t2gfl" (UID: "fff5e63b-e081-4365-9e55-d7a431d73d23") : secret "openshift-state-metrics-tls" not found
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.002989    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrxn7\" (UniqueName: \"kubernetes.io/projected/fff5e63b-e081-4365-9e55-d7a431d73d23-kube-api-access-lrxn7\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.003049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003020    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-wtmp\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.003088    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003096    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-root\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003108    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0902663f-3122-4f56-98a7-770cf809d201-metrics-client-ca\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.003140    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls podName:0902663f-3122-4f56-98a7-770cf809d201 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:47.503124414 +0000 UTC m=+119.315987180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls") pod "node-exporter-jtp4m" (UID: "0902663f-3122-4f56-98a7-770cf809d201") : secret "node-exporter-tls" not found
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0902663f-3122-4f56-98a7-770cf809d201-sys\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003307    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fff5e63b-e081-4365-9e55-d7a431d73d23-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003306    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.003687 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003570    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-accelerators-collector-config\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.004010 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.003743    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.005472 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.005448    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.005954 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.005929    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.006047 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.005960    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.016152 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.016120    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5fc\" (UniqueName: \"kubernetes.io/projected/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-api-access-pj5fc\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.016838 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.016819    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjxnp\" (UniqueName: \"kubernetes.io/projected/0902663f-3122-4f56-98a7-770cf809d201-kube-api-access-wjxnp\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.017191 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.017176    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrxn7\" (UniqueName: \"kubernetes.io/projected/fff5e63b-e081-4365-9e55-d7a431d73d23-kube-api-access-lrxn7\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.507348 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.507306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:47.507811 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.507365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.507811 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.507392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:47.507811 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.507467    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 15:28:47.507811 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.507469    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 06 15:28:47.507811 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.507518    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls podName:0902663f-3122-4f56-98a7-770cf809d201 nodeName:}" failed. No retries permitted until 2026-07-06 15:28:48.507505432 +0000 UTC m=+120.320368188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls") pod "node-exporter-jtp4m" (UID: "0902663f-3122-4f56-98a7-770cf809d201") : secret "node-exporter-tls" not found
Jul 06 15:28:47.507811 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:47.507536    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls podName:8ed5bba2-e1c8-4c56-86a8-1de04638eafa nodeName:}" failed. No retries permitted until 2026-07-06 15:28:48.507522981 +0000 UTC m=+120.320385737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-v6r42" (UID: "8ed5bba2-e1c8-4c56-86a8-1de04638eafa") : secret "kube-state-metrics-tls" not found
Jul 06 15:28:47.510076 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.510047    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fff5e63b-e081-4365-9e55-d7a431d73d23-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-t2gfl\" (UID: \"fff5e63b-e081-4365-9e55-d7a431d73d23\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.715763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.715718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"
Jul 06 15:28:47.871968 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.871935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl"]
Jul 06 15:28:47.875350 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:47.875307    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff5e63b_e081_4365_9e55_d7a431d73d23.slice/crio-0ee7069e09a03c276c0bb6c243e9355fa604b0cf48ec5113bc0c6d230b2b64f8 WatchSource:0}: Error finding container 0ee7069e09a03c276c0bb6c243e9355fa604b0cf48ec5113bc0c6d230b2b64f8: Status 404 returned error can't find the container with id 0ee7069e09a03c276c0bb6c243e9355fa604b0cf48ec5113bc0c6d230b2b64f8
Jul 06 15:28:47.919629 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.919599    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:28:47.924749 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.924722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:47.926891 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.926870    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 15:28:47.927008 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.926910    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 15:28:47.927008 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.926915    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 15:28:47.927332 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.927315    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 15:28:47.927409 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.927345    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 15:28:47.934247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.928249    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 15:28:47.934247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.928303    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 15:28:47.934247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.928616    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 15:28:47.934247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.928991    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tsn7t\""
Jul 06 15:28:47.934247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.929012    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 15:28:47.940650 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:47.940615    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:28:48.011878 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.011841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012023 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.011890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012194 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-web-config\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012238 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012272 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012272 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012353 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-out\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012420 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012397    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nwf8\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-kube-api-access-2nwf8\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012537 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-volume\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.012588 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.012537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113453 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113453 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113453 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113453 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-out\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113453 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nwf8\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-kube-api-access-2nwf8\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-volume\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-web-config\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.113869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.113890    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.115938 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.115175    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.115938 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:28:48.115618    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle podName:29a9fcd3-6c1d-4a65-964b-dc92ba13892e nodeName:}" failed. No retries permitted until 2026-07-06 15:28:48.615593184 +0000 UTC m=+120.428455944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e") : configmap references non-existent config key: ca-bundle.crt
Jul 06 15:28:48.116546 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.116519    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-out\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.117084 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.117057    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.117274 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.117250    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.117335 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.117257    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.117478 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.117456    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.118232 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.118203    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-web-config\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.118318 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.118290    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.118575 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.118554    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.118613 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.118554    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-volume\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.122214 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.122180    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nwf8\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-kube-api-access-2nwf8\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.518406 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.518358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:48.518916 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.518491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:48.521467 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.521426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8ed5bba2-e1c8-4c56-86a8-1de04638eafa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-v6r42\" (UID: \"8ed5bba2-e1c8-4c56-86a8-1de04638eafa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:48.521747 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.521647    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0902663f-3122-4f56-98a7-770cf809d201-node-exporter-tls\") pod \"node-exporter-jtp4m\" (UID: \"0902663f-3122-4f56-98a7-770cf809d201\") " pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:48.591062 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.591023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl" event={"ID":"fff5e63b-e081-4365-9e55-d7a431d73d23","Type":"ContainerStarted","Data":"9066ab6f18a09f01b3921378c8bcee26a8b44d93474bcc645fec9930e2fc07ae"}
Jul 06 15:28:48.591224 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.591100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl" event={"ID":"fff5e63b-e081-4365-9e55-d7a431d73d23","Type":"ContainerStarted","Data":"2f4510947d8f7fe6a682429dbaf32376eb56ca11f6af2028928dab1a1b55449e"}
Jul 06 15:28:48.591224 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.591128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl" event={"ID":"fff5e63b-e081-4365-9e55-d7a431d73d23","Type":"ContainerStarted","Data":"0ee7069e09a03c276c0bb6c243e9355fa604b0cf48ec5113bc0c6d230b2b64f8"}
Jul 06 15:28:48.619592 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.619556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.620474 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.620446    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.644872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.644828    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-p84rw\""
Jul 06 15:28:48.653170 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.653127    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"
Jul 06 15:28:48.653346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.653319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g2nv5\""
Jul 06 15:28:48.661716 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.661671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jtp4m"
Jul 06 15:28:48.675804 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:48.675768    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0902663f_3122_4f56_98a7_770cf809d201.slice/crio-55e0d36b8bf79f81a133db9c154c47aadd43c0e8ae6651cf26365e25bdca7e75 WatchSource:0}: Error finding container 55e0d36b8bf79f81a133db9c154c47aadd43c0e8ae6651cf26365e25bdca7e75: Status 404 returned error can't find the container with id 55e0d36b8bf79f81a133db9c154c47aadd43c0e8ae6651cf26365e25bdca7e75
Jul 06 15:28:48.808767 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.808655    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42"]
Jul 06 15:28:48.840185 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.840154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tsn7t\""
Jul 06 15:28:48.848379 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:48.848346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:28:48.924694 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:48.924665    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed5bba2_e1c8_4c56_86a8_1de04638eafa.slice/crio-1247c3fca5cab0304e6fb25d2ebba3a23d42e485f4d7bf1a2abdde6cf3c141ed WatchSource:0}: Error finding container 1247c3fca5cab0304e6fb25d2ebba3a23d42e485f4d7bf1a2abdde6cf3c141ed: Status 404 returned error can't find the container with id 1247c3fca5cab0304e6fb25d2ebba3a23d42e485f4d7bf1a2abdde6cf3c141ed
Jul 06 15:28:49.086209 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.086003    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:28:49.089041 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:49.089004    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a9fcd3_6c1d_4a65_964b_dc92ba13892e.slice/crio-cf507fcf69596b6e308f9bd8a47965a45fd90ec4d7243d5821d691914fda083d WatchSource:0}: Error finding container cf507fcf69596b6e308f9bd8a47965a45fd90ec4d7243d5821d691914fda083d: Status 404 returned error can't find the container with id cf507fcf69596b6e308f9bd8a47965a45fd90ec4d7243d5821d691914fda083d
Jul 06 15:28:49.597011 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.596884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42" event={"ID":"8ed5bba2-e1c8-4c56-86a8-1de04638eafa","Type":"ContainerStarted","Data":"1247c3fca5cab0304e6fb25d2ebba3a23d42e485f4d7bf1a2abdde6cf3c141ed"}
Jul 06 15:28:49.598458 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.598425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtp4m" event={"ID":"0902663f-3122-4f56-98a7-770cf809d201","Type":"ContainerStarted","Data":"55e0d36b8bf79f81a133db9c154c47aadd43c0e8ae6651cf26365e25bdca7e75"}
Jul 06 15:28:49.599884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.599845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"cf507fcf69596b6e308f9bd8a47965a45fd90ec4d7243d5821d691914fda083d"}
Jul 06 15:28:49.602235 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.602168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl" event={"ID":"fff5e63b-e081-4365-9e55-d7a431d73d23","Type":"ContainerStarted","Data":"c1a13f7815da151632c4c3f7d904beaa1794bc74b5426b175ed588a66df501e4"}
Jul 06 15:28:49.625542 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.625484    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-t2gfl" podStartSLOduration=2.651429768 podStartE2EDuration="3.625465364s" totalImagesPullingTime="974.035596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:49.624776134 +0000 UTC m=+121.437638931" watchObservedRunningTime="2026-07-06 15:28:49.625465364 +0000 UTC m=+121.438328142"
Jul 06 15:28:49.873555 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.872825    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"]
Jul 06 15:28:49.877683 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.877655    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.881268 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 06 15:28:49.881268 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 06 15:28:49.881519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881312    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-lcouefuckrjp\""
Jul 06 15:28:49.881519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 06 15:28:49.881519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881377    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 06 15:28:49.881519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881470    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-s4qzt\""
Jul 06 15:28:49.881519 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.881499    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 06 15:28:49.901509 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.901476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"]
Jul 06 15:28:49.932286 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-tls\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-grpc-tls\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbbqb\" (UniqueName: \"kubernetes.io/projected/c2000c95-00d3-4b24-a9f6-d47fea3a3030-kube-api-access-qbbqb\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2000c95-00d3-4b24-a9f6-d47fea3a3030-metrics-client-ca\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:49.932932 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:49.932620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034067 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034256 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034256 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034256 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-grpc-tls\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034256 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbbqb\" (UniqueName: \"kubernetes.io/projected/c2000c95-00d3-4b24-a9f6-d47fea3a3030-kube-api-access-qbbqb\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034256 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2000c95-00d3-4b24-a9f6-d47fea3a3030-metrics-client-ca\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034507 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.034507 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.034319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-tls\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.035216 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.035156    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2000c95-00d3-4b24-a9f6-d47fea3a3030-metrics-client-ca\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.037866 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.037810    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-grpc-tls\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.037866 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.037822    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.037866 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.037808    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.038373 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.038196    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.038373 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.038314    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-tls\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.038542 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.038523    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2000c95-00d3-4b24-a9f6-d47fea3a3030-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.045205 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.045172    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbbqb\" (UniqueName: \"kubernetes.io/projected/c2000c95-00d3-4b24-a9f6-d47fea3a3030-kube-api-access-qbbqb\") pod \"thanos-querier-5f74cb6d84-vbh6m\" (UID: \"c2000c95-00d3-4b24-a9f6-d47fea3a3030\") " pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.204528 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.204489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:50.608355 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.608275    2580 generic.go:356] "Generic (PLEG): container finished" podID="0902663f-3122-4f56-98a7-770cf809d201" containerID="ea59c2ce60d2773fa75f96e7d62d398fbcc630832bdea6e600770fe0f11e714b" exitCode=0
Jul 06 15:28:50.608766 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.608361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtp4m" event={"ID":"0902663f-3122-4f56-98a7-770cf809d201","Type":"ContainerDied","Data":"ea59c2ce60d2773fa75f96e7d62d398fbcc630832bdea6e600770fe0f11e714b"}
Jul 06 15:28:50.797466 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:50.797341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"]
Jul 06 15:28:50.801049 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:50.801011    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2000c95_00d3_4b24_a9f6_d47fea3a3030.slice/crio-9a4c19ad48c1df31a317c549ac4c7922f0a12d4a494d260a0819dabb17870394 WatchSource:0}: Error finding container 9a4c19ad48c1df31a317c549ac4c7922f0a12d4a494d260a0819dabb17870394: Status 404 returned error can't find the container with id 9a4c19ad48c1df31a317c549ac4c7922f0a12d4a494d260a0819dabb17870394
Jul 06 15:28:51.131759 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.131720    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75cd77d48f-cv6t8"]
Jul 06 15:28:51.135023 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.134994    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.137680 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.137647    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 06 15:28:51.137680 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.137647    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-20rrijm17017m\""
Jul 06 15:28:51.138018 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.138002    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 15:28:51.138393 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.138374    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 06 15:28:51.138450 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.138432    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 06 15:28:51.138496 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.138452    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jm2cq\""
Jul 06 15:28:51.151483 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.151451    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75cd77d48f-cv6t8"]
Jul 06 15:28:51.246779 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.246734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-secret-metrics-server-client-certs\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.246948 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.246847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0397f69d-33f9-4bed-be4e-4cbe1497574f-audit-log\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.246948 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.246923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-client-ca-bundle\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.247047 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.246969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-secret-metrics-server-tls\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.247099 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.247054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0397f69d-33f9-4bed-be4e-4cbe1497574f-metrics-server-audit-profiles\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.247153 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.247096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0397f69d-33f9-4bed-be4e-4cbe1497574f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.247153 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.247124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26cn\" (UniqueName: \"kubernetes.io/projected/0397f69d-33f9-4bed-be4e-4cbe1497574f-kube-api-access-v26cn\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.347770 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0397f69d-33f9-4bed-be4e-4cbe1497574f-audit-log\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.347946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-client-ca-bundle\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.347946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-secret-metrics-server-tls\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.347946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0397f69d-33f9-4bed-be4e-4cbe1497574f-metrics-server-audit-profiles\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.347946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0397f69d-33f9-4bed-be4e-4cbe1497574f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.347946 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26cn\" (UniqueName: \"kubernetes.io/projected/0397f69d-33f9-4bed-be4e-4cbe1497574f-kube-api-access-v26cn\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.348200 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.347956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-secret-metrics-server-client-certs\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.348250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.348235    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0397f69d-33f9-4bed-be4e-4cbe1497574f-audit-log\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.348886 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.348840    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0397f69d-33f9-4bed-be4e-4cbe1497574f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.349280 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.349231    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0397f69d-33f9-4bed-be4e-4cbe1497574f-metrics-server-audit-profiles\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.351168 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.351144    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-secret-metrics-server-client-certs\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.351305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.351267    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-secret-metrics-server-tls\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.351677 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.351653    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0397f69d-33f9-4bed-be4e-4cbe1497574f-client-ca-bundle\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.368830 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.368802    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26cn\" (UniqueName: \"kubernetes.io/projected/0397f69d-33f9-4bed-be4e-4cbe1497574f-kube-api-access-v26cn\") pod \"metrics-server-75cd77d48f-cv6t8\" (UID: \"0397f69d-33f9-4bed-be4e-4cbe1497574f\") " pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.446869 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.446757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:28:51.616948 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.616914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"9a4c19ad48c1df31a317c549ac4c7922f0a12d4a494d260a0819dabb17870394"}
Jul 06 15:28:51.618595 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.618563    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201" exitCode=0
Jul 06 15:28:51.618781 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.618659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201"}
Jul 06 15:28:51.621724 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.621669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42" event={"ID":"8ed5bba2-e1c8-4c56-86a8-1de04638eafa","Type":"ContainerStarted","Data":"015854d0d67b86dccd80588ff80be12d8c9deeea1c90fa70123fb24efaab9dee"}
Jul 06 15:28:51.621843 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.621768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42" event={"ID":"8ed5bba2-e1c8-4c56-86a8-1de04638eafa","Type":"ContainerStarted","Data":"384f96e77699033e2d988618f629959a605a08953af2e3c99d1b7e80c22032cc"}
Jul 06 15:28:51.621843 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.621793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42" event={"ID":"8ed5bba2-e1c8-4c56-86a8-1de04638eafa","Type":"ContainerStarted","Data":"7365cc6473f9ca9e5e8c38ec8e396637f35916629f7ddab505c303f6ad975888"}
Jul 06 15:28:51.624410 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.624361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtp4m" event={"ID":"0902663f-3122-4f56-98a7-770cf809d201","Type":"ContainerStarted","Data":"e6c1420ecdd7bff4b26c0e81a75f47a975c67dde7bea82e7719e2cd5e2e64da8"}
Jul 06 15:28:51.624525 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.624439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jtp4m" event={"ID":"0902663f-3122-4f56-98a7-770cf809d201","Type":"ContainerStarted","Data":"e8ce314ce1a1a86f5db8b7e3f531831541deb187fcf1678069a3ac6757a48aa9"}
Jul 06 15:28:51.635166 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.635129    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75cd77d48f-cv6t8"]
Jul 06 15:28:51.639716 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:51.639659    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0397f69d_33f9_4bed_be4e_4cbe1497574f.slice/crio-cbbc07bad097f81889b84d8bebed9f48d6a592c368062caa5380dc0caa174683 WatchSource:0}: Error finding container cbbc07bad097f81889b84d8bebed9f48d6a592c368062caa5380dc0caa174683: Status 404 returned error can't find the container with id cbbc07bad097f81889b84d8bebed9f48d6a592c368062caa5380dc0caa174683
Jul 06 15:28:51.698379 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.698235    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-v6r42" podStartSLOduration=3.993315244 podStartE2EDuration="5.69821322s" totalImagesPullingTime="1.704897976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:51.69766005 +0000 UTC m=+123.510522841" watchObservedRunningTime="2026-07-06 15:28:51.69821322 +0000 UTC m=+123.511076000"
Jul 06 15:28:51.729434 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:51.729306    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jtp4m" podStartSLOduration=4.6505955960000005 podStartE2EDuration="5.729283577s" totalImagesPullingTime="983.613326ms" totalInitContainerRuntime="95.074655ms" isStatefulPod=true podCreationTimestamp="2026-07-06 15:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:51.728622871 +0000 UTC m=+123.541485672" watchObservedRunningTime="2026-07-06 15:28:51.729283577 +0000 UTC m=+123.542146356"
Jul 06 15:28:52.629349 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:52.629296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8" event={"ID":"0397f69d-33f9-4bed-be4e-4cbe1497574f","Type":"ContainerStarted","Data":"cbbc07bad097f81889b84d8bebed9f48d6a592c368062caa5380dc0caa174683"}
Jul 06 15:28:53.636245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:53.636197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"9dcfe401c010f50b7ebeb1fcf107e7573e35f64029cd5b4242e9133b19128140"}
Jul 06 15:28:53.636759 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:53.636311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"8ff45a78ccf28ddfdbac646f8d10661d1bfd3ec2b216d4b95d69735c53e5d7dd"}
Jul 06 15:28:53.636759 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:53.636348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"f8b0134607c6b49bd69a5eed9a80d03facfea24e8cf11ad6602ca814a5336d15"}
Jul 06 15:28:54.647664 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.647622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"9ef153143bf7110ab219da57ea24d342806270aa015c03e546aa56454808eed8"}
Jul 06 15:28:54.648087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.647772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:28:54.648087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.647830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"b790444294ea7aaf65e2df5c92e47c867e468a0878023a9e49f68a00a4384910"}
Jul 06 15:28:54.648087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.647890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" event={"ID":"c2000c95-00d3-4b24-a9f6-d47fea3a3030","Type":"ContainerStarted","Data":"c64c01328d069c0c32bb2e520e4892afa08f8bd50ce231e09a701f921061b4a0"}
Jul 06 15:28:54.651086 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.651058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b"}
Jul 06 15:28:54.651214 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.651131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b"}
Jul 06 15:28:54.651214 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.651157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d"}
Jul 06 15:28:54.651214 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.651203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c"}
Jul 06 15:28:54.651330 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.651227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80"}
Jul 06 15:28:54.652954 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.652921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8" event={"ID":"0397f69d-33f9-4bed-be4e-4cbe1497574f","Type":"ContainerStarted","Data":"6ae595a7816a6921fd11a432d365ca18eba48af654724a0b8ec60642aaab090c"}
Jul 06 15:28:54.674479 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.674407    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m" podStartSLOduration=2.46730986 podStartE2EDuration="5.674387862s" totalImagesPullingTime="3.207078002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:54.672666842 +0000 UTC m=+126.485529631" watchObservedRunningTime="2026-07-06 15:28:54.674387862 +0000 UTC m=+126.487250640"
Jul 06 15:28:54.695772 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:54.695693    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8" podStartSLOduration=1.097233217 podStartE2EDuration="3.695673441s" totalImagesPullingTime="2.598440224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:54.691876751 +0000 UTC m=+126.504739542" watchObservedRunningTime="2026-07-06 15:28:54.695673441 +0000 UTC m=+126.508536219"
Jul 06 15:28:55.660786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:55.660737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerStarted","Data":"94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862"}
Jul 06 15:28:55.692446 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:55.692383    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.511943483 podStartE2EDuration="8.692364562s" totalImagesPullingTime="4.157466958s" totalInitContainerRuntime="22.954121ms" isStatefulPod=false podCreationTimestamp="2026-07-06 15:28:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:28:55.690948375 +0000 UTC m=+127.503811153" watchObservedRunningTime="2026-07-06 15:28:55.692364562 +0000 UTC m=+127.505227341"
Jul 06 15:28:56.423291 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:56.423257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:28:58.427157 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:58.427118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:28:58.429586 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:58.429561    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fd83a4d-6fff-4447-a374-13e06a499206-metrics-certs\") pod \"network-metrics-daemon-rlzvq\" (UID: \"9fd83a4d-6fff-4447-a374-13e06a499206\") " pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:28:58.638071 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:58.638036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-km2tx\""
Jul 06 15:28:58.645899 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:58.645858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rlzvq"
Jul 06 15:28:58.789582 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:58.789545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rlzvq"]
Jul 06 15:28:58.793108 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:28:58.793080    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd83a4d_6fff_4447_a374_13e06a499206.slice/crio-ff8599a094f9acf43b33caf917b3e7b5f5414db9a3807634d89bc6ce9ad2ecaf WatchSource:0}: Error finding container ff8599a094f9acf43b33caf917b3e7b5f5414db9a3807634d89bc6ce9ad2ecaf: Status 404 returned error can't find the container with id ff8599a094f9acf43b33caf917b3e7b5f5414db9a3807634d89bc6ce9ad2ecaf
Jul 06 15:28:59.681470 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:28:59.681436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rlzvq" event={"ID":"9fd83a4d-6fff-4447-a374-13e06a499206","Type":"ContainerStarted","Data":"ff8599a094f9acf43b33caf917b3e7b5f5414db9a3807634d89bc6ce9ad2ecaf"}
Jul 06 15:29:00.667653 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:00.667626    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f74cb6d84-vbh6m"
Jul 06 15:29:00.687403 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:00.687366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rlzvq" event={"ID":"9fd83a4d-6fff-4447-a374-13e06a499206","Type":"ContainerStarted","Data":"ff1a318f4ad0a1c6c404e6f1d28d8255bfd262dc2ccfc3623e64bad7bd4d4d95"}
Jul 06 15:29:00.687788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:00.687429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rlzvq" event={"ID":"9fd83a4d-6fff-4447-a374-13e06a499206","Type":"ContainerStarted","Data":"a10721e6e28ecc1d63701a9b8f76d3a403edb247827344bf2a66179ed966f737"}
Jul 06 15:29:00.710414 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:00.710339    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rlzvq" podStartSLOduration=130.818116456 podStartE2EDuration="2m11.71031536s" totalImagesPullingTime="892.198904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:29:00.707930552 +0000 UTC m=+132.520793330" watchObservedRunningTime="2026-07-06 15:29:00.71031536 +0000 UTC m=+132.523178142"
Jul 06 15:29:01.438566 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.438519    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerName="registry" containerID="cri-o://4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd" gracePeriod=30
Jul 06 15:29:01.693182 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.693108    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:29:01.693182 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.693120    2580 generic.go:356] "Generic (PLEG): container finished" podID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerID="4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd" exitCode=0
Jul 06 15:29:01.693618 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.693193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" event={"ID":"1d200281-ce2e-445e-b6d2-bdf380db6d25","Type":"ContainerDied","Data":"4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd"}
Jul 06 15:29:01.693618 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.693247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-646b498d44-b5d4g" event={"ID":"1d200281-ce2e-445e-b6d2-bdf380db6d25","Type":"ContainerDied","Data":"040f4556b39082c25e8ed47318417f9587e41708daa9c96d8a3f97387eec40cc"}
Jul 06 15:29:01.693618 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.693270    2580 scope.go:117] "RemoveContainer" containerID="4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd"
Jul 06 15:29:01.704384 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.704355    2580 scope.go:117] "RemoveContainer" containerID="4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd"
Jul 06 15:29:01.704785 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:29:01.704757    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd\": container with ID starting with 4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd not found: ID does not exist" containerID="4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd"
Jul 06 15:29:01.704845 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.704803    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd"} err="failed to get container status \"4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd\": rpc error: code = NotFound desc = could not find container \"4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd\": container with ID starting with 4501c95733447721ad3237ab5fa1010ae034ccee3ec3fb86d7aea3cdcfeeb0dd not found: ID does not exist"
Jul 06 15:29:01.759502 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-installation-pull-secrets\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759502 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759512    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-certificates\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d200281-ce2e-445e-b6d2-bdf380db6d25-ca-trust-extracted\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-trusted-ca\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759633    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-image-registry-private-configuration\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759666    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-bound-sa-token\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759724    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.759788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.759752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqg9z\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-kube-api-access-lqg9z\") pod \"1d200281-ce2e-445e-b6d2-bdf380db6d25\" (UID: \"1d200281-ce2e-445e-b6d2-bdf380db6d25\") "
Jul 06 15:29:01.760085 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.760049    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:29:01.760158 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.760133    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:29:01.762507 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.762465    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:29:01.762507 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.762487    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:29:01.762765 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.762652    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-kube-api-access-lqg9z" (OuterVolumeSpecName: "kube-api-access-lqg9z") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "kube-api-access-lqg9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:29:01.762765 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.762693    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:29:01.762867 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.762825    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:29:01.768825 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.768794    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d200281-ce2e-445e-b6d2-bdf380db6d25-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1d200281-ce2e-445e-b6d2-bdf380db6d25" (UID: "1d200281-ce2e-445e-b6d2-bdf380db6d25"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:29:01.860624 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860588    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-tls\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860624 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860620    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqg9z\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-kube-api-access-lqg9z\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860624 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860630    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-installation-pull-secrets\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860881 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860640    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-registry-certificates\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860881 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860649    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1d200281-ce2e-445e-b6d2-bdf380db6d25-ca-trust-extracted\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860881 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860659    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1d200281-ce2e-445e-b6d2-bdf380db6d25-trusted-ca\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860881 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860668    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1d200281-ce2e-445e-b6d2-bdf380db6d25-image-registry-private-configuration\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:01.860881 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:01.860676    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d200281-ce2e-445e-b6d2-bdf380db6d25-bound-sa-token\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:29:02.237135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.237098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fcfdbbff8-2ldrh"]
Jul 06 15:29:02.237436 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.237423    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerName="registry"
Jul 06 15:29:02.237436 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.237437    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerName="registry"
Jul 06 15:29:02.237509 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.237497    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" containerName="registry"
Jul 06 15:29:02.264559 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.264525    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcfdbbff8-2ldrh"]
Jul 06 15:29:02.264767 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.264630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.267457 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.267422    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 15:29:02.268247 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.268211    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rmsbt\""
Jul 06 15:29:02.268384 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.268276    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 15:29:02.268384 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.268301    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 15:29:02.268384 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.268353    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 15:29:02.268536 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.268503    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 15:29:02.268639 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.268615    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 15:29:02.269021 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.269003    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 15:29:02.278009 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.277980    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 15:29:02.365510 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-oauth-serving-cert\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.365510 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-config\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.365788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-service-ca\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.365788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-trusted-ca-bundle\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.365788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-serving-cert\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.365788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-oauth-config\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.365788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.365752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp4kl\" (UniqueName: \"kubernetes.io/projected/0191ee87-032b-4c01-b3e3-d7552d83bcad-kube-api-access-tp4kl\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466428 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-oauth-config\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466608 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp4kl\" (UniqueName: \"kubernetes.io/projected/0191ee87-032b-4c01-b3e3-d7552d83bcad-kube-api-access-tp4kl\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466608 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-oauth-serving-cert\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466608 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-config\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466608 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-service-ca\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466608 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-trusted-ca-bundle\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.466917 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.466615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-serving-cert\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.467376 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.467347    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-oauth-serving-cert\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.467376 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.467370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-config\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.467572 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.467527    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-service-ca\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.467630 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.467594    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-trusted-ca-bundle\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.469123 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.469100    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-serving-cert\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.469366 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.469341    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-oauth-config\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.475466 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.475434    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp4kl\" (UniqueName: \"kubernetes.io/projected/0191ee87-032b-4c01-b3e3-d7552d83bcad-kube-api-access-tp4kl\") pod \"console-5fcfdbbff8-2ldrh\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") " pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.583806 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.583770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:02.702765 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.702188    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-646b498d44-b5d4g"
Jul 06 15:29:02.724589 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.724546    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-646b498d44-b5d4g"]
Jul 06 15:29:02.728870 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.728831    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-646b498d44-b5d4g"]
Jul 06 15:29:02.792953 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:02.792781    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcfdbbff8-2ldrh"]
Jul 06 15:29:02.795753 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:29:02.795724    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0191ee87_032b_4c01_b3e3_d7552d83bcad.slice/crio-4dd62d26518c5de9cf2adfb161936eb04ea5afd25e177cea5db743e0cc574c9b WatchSource:0}: Error finding container 4dd62d26518c5de9cf2adfb161936eb04ea5afd25e177cea5db743e0cc574c9b: Status 404 returned error can't find the container with id 4dd62d26518c5de9cf2adfb161936eb04ea5afd25e177cea5db743e0cc574c9b
Jul 06 15:29:03.709852 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:03.709816    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfdbbff8-2ldrh" event={"ID":"0191ee87-032b-4c01-b3e3-d7552d83bcad","Type":"ContainerStarted","Data":"4dd62d26518c5de9cf2adfb161936eb04ea5afd25e177cea5db743e0cc574c9b"}
Jul 06 15:29:04.021262 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:04.021178    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d200281-ce2e-445e-b6d2-bdf380db6d25" path="/var/lib/kubelet/pods/1d200281-ce2e-445e-b6d2-bdf380db6d25/volumes"
Jul 06 15:29:06.723850 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:06.723812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfdbbff8-2ldrh" event={"ID":"0191ee87-032b-4c01-b3e3-d7552d83bcad","Type":"ContainerStarted","Data":"9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb"}
Jul 06 15:29:06.744576 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:06.744509    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fcfdbbff8-2ldrh" podStartSLOduration=1.6820859270000001 podStartE2EDuration="4.744485583s" totalImagesPullingTime="3.062399656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:29:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:29:06.743788498 +0000 UTC m=+138.556651276" watchObservedRunningTime="2026-07-06 15:29:06.744485583 +0000 UTC m=+138.557348361"
Jul 06 15:29:11.447282 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:11.447237    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:29:11.447784 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:11.447320    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:29:12.584122 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:12.584080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:12.584504 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:12.584162    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:12.588895 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:12.588874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:12.753266 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:12.753240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:29:31.452994 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:31.452961    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:29:31.456844 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:29:31.456821    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75cd77d48f-cv6t8"
Jul 06 15:30:07.348385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348346    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:30:07.348986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348750    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="alertmanager" containerID="cri-o://4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80" gracePeriod=120
Jul 06 15:30:07.348986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348792    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-metric" containerID="cri-o://822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b" gracePeriod=120
Jul 06 15:30:07.348986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348816    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-web" containerID="cri-o://9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d" gracePeriod=120
Jul 06 15:30:07.348986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348862    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="config-reloader" containerID="cri-o://634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c" gracePeriod=120
Jul 06 15:30:07.348986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348870    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy" containerID="cri-o://2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b" gracePeriod=120
Jul 06 15:30:07.348986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.348887    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="prom-label-proxy" containerID="cri-o://94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862" gracePeriod=120
Jul 06 15:30:07.984036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.983995    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862" exitCode=0
Jul 06 15:30:07.984036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984023    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b" exitCode=0
Jul 06 15:30:07.984036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984029    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d" exitCode=0
Jul 06 15:30:07.984036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984035    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c" exitCode=0
Jul 06 15:30:07.984036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984043    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80" exitCode=0
Jul 06 15:30:07.984323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984069    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862"}
Jul 06 15:30:07.984323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b"}
Jul 06 15:30:07.984323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d"}
Jul 06 15:30:07.984323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c"}
Jul 06 15:30:07.984323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:07.984230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80"}
Jul 06 15:30:08.594171 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.594142    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:08.724552 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-web\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724552 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-metrics-client-ca\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724552 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-web-config\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724825 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724654    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nwf8\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-kube-api-access-2nwf8\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724825 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724825 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-main-db\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724902    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-tls-assets\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724938    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-out\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.724986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724971    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-main-tls\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.725249 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724986    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:30:08.725249 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.724999    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-cluster-tls-config\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.725249 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.725040    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-volume\") pod \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\" (UID: \"29a9fcd3-6c1d-4a65-964b-dc92ba13892e\") "
Jul 06 15:30:08.725405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.725245    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:30:08.725405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.725304    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-metrics-client-ca\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.726405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.726374    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:30:08.727790 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.727545    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.727790 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.727689    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:30:08.728135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.728106    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.728532 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.728500    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-kube-api-access-2nwf8" (OuterVolumeSpecName: "kube-api-access-2nwf8") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "kube-api-access-2nwf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:30:08.728631 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.728528    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-volume" (OuterVolumeSpecName: "config-volume") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.728690 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.728658    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.729308 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.729284    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-out" (OuterVolumeSpecName: "config-out") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:30:08.729450 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.729433    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.732678 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.732565    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.740260 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.740220    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-web-config" (OuterVolumeSpecName: "web-config") pod "29a9fcd3-6c1d-4a65-964b-dc92ba13892e" (UID: "29a9fcd3-6c1d-4a65-964b-dc92ba13892e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:08.825995 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.825952    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-volume\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.825995 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.825988    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.825995 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826000    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-web-config\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826010    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nwf8\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-kube-api-access-2nwf8\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826020    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826029    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826038    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-alertmanager-main-db\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826049    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826057    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-tls-assets\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826065    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-config-out\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826073    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-secret-alertmanager-main-tls\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.826242 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.826081    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/29a9fcd3-6c1d-4a65-964b-dc92ba13892e-cluster-tls-config\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:08.991551 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.991465    2580 generic.go:356] "Generic (PLEG): container finished" podID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerID="822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b" exitCode=0
Jul 06 15:30:08.991689 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.991549    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:08.991689 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.991546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b"}
Jul 06 15:30:08.991786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.991721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"29a9fcd3-6c1d-4a65-964b-dc92ba13892e","Type":"ContainerDied","Data":"cf507fcf69596b6e308f9bd8a47965a45fd90ec4d7243d5821d691914fda083d"}
Jul 06 15:30:08.991786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.991757    2580 scope.go:117] "RemoveContainer" containerID="94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862"
Jul 06 15:30:08.999877 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:08.999851    2580 scope.go:117] "RemoveContainer" containerID="822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b"
Jul 06 15:30:09.007467 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.007443    2580 scope.go:117] "RemoveContainer" containerID="2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b"
Jul 06 15:30:09.015029 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.015007    2580 scope.go:117] "RemoveContainer" containerID="9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d"
Jul 06 15:30:09.017921 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.017892    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:30:09.023685 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.023630    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:30:09.024333 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.024317    2580 scope.go:117] "RemoveContainer" containerID="634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c"
Jul 06 15:30:09.032045 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.032021    2580 scope.go:117] "RemoveContainer" containerID="4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80"
Jul 06 15:30:09.039877 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.039852    2580 scope.go:117] "RemoveContainer" containerID="68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201"
Jul 06 15:30:09.047723 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.047678    2580 scope.go:117] "RemoveContainer" containerID="94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862"
Jul 06 15:30:09.048025 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.048007    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862\": container with ID starting with 94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862 not found: ID does not exist" containerID="94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862"
Jul 06 15:30:09.048070 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048038    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862"} err="failed to get container status \"94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862\": rpc error: code = NotFound desc = could not find container \"94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862\": container with ID starting with 94be43b59a8d3a340032dc0de93a6ea88dd5de40708416f24345063eeab1e862 not found: ID does not exist"
Jul 06 15:30:09.048070 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048063    2580 scope.go:117] "RemoveContainer" containerID="822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b"
Jul 06 15:30:09.048325 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.048310    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b\": container with ID starting with 822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b not found: ID does not exist" containerID="822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b"
Jul 06 15:30:09.048366 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048329    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b"} err="failed to get container status \"822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b\": rpc error: code = NotFound desc = could not find container \"822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b\": container with ID starting with 822f11409e4ab39e2cc5a4e1d5133abb1706690ff71efff832fa7d55c16df08b not found: ID does not exist"
Jul 06 15:30:09.048366 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048341    2580 scope.go:117] "RemoveContainer" containerID="2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b"
Jul 06 15:30:09.048548 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.048526    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b\": container with ID starting with 2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b not found: ID does not exist" containerID="2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b"
Jul 06 15:30:09.048590 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048560    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b"} err="failed to get container status \"2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b\": rpc error: code = NotFound desc = could not find container \"2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b\": container with ID starting with 2d830b48b8588f3e77ae4a7d43861b6e2ed4accdd48300b7ff06ef127b68e38b not found: ID does not exist"
Jul 06 15:30:09.048590 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048580    2580 scope.go:117] "RemoveContainer" containerID="9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d"
Jul 06 15:30:09.048943 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.048923    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d\": container with ID starting with 9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d not found: ID does not exist" containerID="9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d"
Jul 06 15:30:09.049045 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048943    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d"} err="failed to get container status \"9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d\": rpc error: code = NotFound desc = could not find container \"9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d\": container with ID starting with 9844a1546b59d9fe0cf59dacbaf3b55da9d803a720a70f17d80f9847d840456d not found: ID does not exist"
Jul 06 15:30:09.049045 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.048957    2580 scope.go:117] "RemoveContainer" containerID="634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c"
Jul 06 15:30:09.049214 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.049193    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c\": container with ID starting with 634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c not found: ID does not exist" containerID="634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c"
Jul 06 15:30:09.049263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.049220    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c"} err="failed to get container status \"634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c\": rpc error: code = NotFound desc = could not find container \"634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c\": container with ID starting with 634ee723f4e8d404e8280603916968976275d12615fe16be41eeb507d0d3d80c not found: ID does not exist"
Jul 06 15:30:09.049263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.049236    2580 scope.go:117] "RemoveContainer" containerID="4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80"
Jul 06 15:30:09.049462 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.049445    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80\": container with ID starting with 4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80 not found: ID does not exist" containerID="4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80"
Jul 06 15:30:09.049510 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.049467    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80"} err="failed to get container status \"4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80\": rpc error: code = NotFound desc = could not find container \"4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80\": container with ID starting with 4afd23b1e73d8f41e8a92eeccf8d6c68da481d911962dc23ef60688259132b80 not found: ID does not exist"
Jul 06 15:30:09.049510 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.049481    2580 scope.go:117] "RemoveContainer" containerID="68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201"
Jul 06 15:30:09.049754 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:30:09.049737    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201\": container with ID starting with 68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201 not found: ID does not exist" containerID="68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201"
Jul 06 15:30:09.049813 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.049758    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201"} err="failed to get container status \"68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201\": rpc error: code = NotFound desc = could not find container \"68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201\": container with ID starting with 68daddcb4c632c027a54c327936f3fc9364c5bb535b5b24dce0d5092d0edb201 not found: ID does not exist"
Jul 06 15:30:09.067974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.067941    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:30:09.068250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068238    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="alertmanager"
Jul 06 15:30:09.068292 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068251    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="alertmanager"
Jul 06 15:30:09.068292 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068260    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-metric"
Jul 06 15:30:09.068292 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068266    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-metric"
Jul 06 15:30:09.068292 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068275    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="config-reloader"
Jul 06 15:30:09.068292 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068280    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="config-reloader"
Jul 06 15:30:09.068292 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068290    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="init-config-reloader"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068295    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="init-config-reloader"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068303    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="prom-label-proxy"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068308    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="prom-label-proxy"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068320    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068325    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068333    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-web"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068339    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-web"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068399    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068408    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="config-reloader"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068413    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="prom-label-proxy"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068419    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-metric"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068424    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="alertmanager"
Jul 06 15:30:09.068461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.068430    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" containerName="kube-rbac-proxy-web"
Jul 06 15:30:09.073411 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.073391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.075948 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.075919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 06 15:30:09.076084 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.075920    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 06 15:30:09.076084 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.075920    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 06 15:30:09.076084 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.075919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 06 15:30:09.076084 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.076071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 06 15:30:09.077820 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.077799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 06 15:30:09.077974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.077955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 06 15:30:09.078018 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.078006    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 06 15:30:09.078583 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.078568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tsn7t\""
Jul 06 15:30:09.084571 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.084546    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 06 15:30:09.093150 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.093120    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:30:09.229029 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.228987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229029 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhx2h\" (UniqueName: \"kubernetes.io/projected/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-kube-api-access-nhx2h\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-web-config\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-config-out\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.229562 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.229426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhx2h\" (UniqueName: \"kubernetes.io/projected/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-kube-api-access-nhx2h\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330462 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-web-config\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330462 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330462 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-config-out\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330670 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330806 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330806 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330806 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.330986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.330869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.331816 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.331788    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.333059 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.333030    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.333339 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.333316    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.333455 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.333376    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.333750 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.333726    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-web-config\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334130 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334107    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-config-out\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334130 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334121    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334317 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334295    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334355 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334306    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334355 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334336    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334416 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334400    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.334923 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.334906    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.343027 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.342995    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhx2h\" (UniqueName: \"kubernetes.io/projected/4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff-kube-api-access-nhx2h\") pod \"alertmanager-main-0\" (UID: \"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.384006 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.383939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 06 15:30:09.534941 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.534915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 06 15:30:09.537544 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:30:09.537503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5e56ed_33df_494e_aa43_d9aeaa0bc1ff.slice/crio-32b31cdd477dadac436c190c1a180e07bc64aa1e157ebda36aee926d17dc8769 WatchSource:0}: Error finding container 32b31cdd477dadac436c190c1a180e07bc64aa1e157ebda36aee926d17dc8769: Status 404 returned error can't find the container with id 32b31cdd477dadac436c190c1a180e07bc64aa1e157ebda36aee926d17dc8769
Jul 06 15:30:09.998678 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.998642    2580 generic.go:356] "Generic (PLEG): container finished" podID="4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff" containerID="49b3094654961495ea8f922700383328dbefdd163082681ffae138f559936472" exitCode=0
Jul 06 15:30:09.999149 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.998720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerDied","Data":"49b3094654961495ea8f922700383328dbefdd163082681ffae138f559936472"}
Jul 06 15:30:09.999149 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:09.998770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"32b31cdd477dadac436c190c1a180e07bc64aa1e157ebda36aee926d17dc8769"}
Jul 06 15:30:10.021153 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:10.021057    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29a9fcd3-6c1d-4a65-964b-dc92ba13892e" path="/var/lib/kubelet/pods/29a9fcd3-6c1d-4a65-964b-dc92ba13892e/volumes"
Jul 06 15:30:11.006902 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.006864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"bb6c76bfaf829418cd2119c8d73f2c15080457104dd1d3dacea7b4d297658d86"}
Jul 06 15:30:11.007375 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.006988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"f017a6705cafc4fc9b1ae705a6a4adefa8c5ebb279ebc317f50dcb312d4711d8"}
Jul 06 15:30:11.007375 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.007033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"3bd26c0f3c20a14d4b14e64663928a7f3a15f453202c42ea4ab4175aa032d6ae"}
Jul 06 15:30:11.007375 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.007082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"544c0fbebc02655d6f61d6a80bbd52e232970dabcfe258dea8637c9b346bb8cb"}
Jul 06 15:30:11.007375 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.007121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"df4584d59a003a9df0fd106d4b2e5dab1afa88c77648e48c10b7b9537bc224aa"}
Jul 06 15:30:11.007375 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.007162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff","Type":"ContainerStarted","Data":"28aee37270518f7bbd0ac43eda4e4191bb1eb1929b10d9d1ead1e73880533590"}
Jul 06 15:30:11.057264 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:11.057211    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.04169626 podStartE2EDuration="2.057195134s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.498874ms" isStatefulPod=false podCreationTimestamp="2026-07-06 15:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:30:11.056008597 +0000 UTC m=+202.868871375" watchObservedRunningTime="2026-07-06 15:30:11.057195134 +0000 UTC m=+202.870057912"
Jul 06 15:30:23.536318 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.536284    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78bb99c546-zzlvc"]
Jul 06 15:30:23.540154 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.540129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.556555 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.556519    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78bb99c546-zzlvc"]
Jul 06 15:30:23.650438 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-service-ca\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.650438 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-oauth-serving-cert\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.650661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-serving-cert\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.650661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-oauth-config\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.650661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-trusted-ca-bundle\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.650661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-config\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.650661 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.650575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8dlv\" (UniqueName: \"kubernetes.io/projected/3b9f36ac-464d-4f32-bfec-6526fbe7640b-kube-api-access-s8dlv\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.751610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-config\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.751691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8dlv\" (UniqueName: \"kubernetes.io/projected/3b9f36ac-464d-4f32-bfec-6526fbe7640b-kube-api-access-s8dlv\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.751802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-service-ca\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.751840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-oauth-serving-cert\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.751973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-serving-cert\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.752031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-oauth-config\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752385 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.752101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-trusted-ca-bundle\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752865 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.752785    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-config\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.752865 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.752809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-oauth-serving-cert\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.757831 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.753901    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-trusted-ca-bundle\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.757831 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.754023    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-service-ca\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.757831 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.756031    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-oauth-config\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.758844 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.758822    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-serving-cert\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.768790 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.768757    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8dlv\" (UniqueName: \"kubernetes.io/projected/3b9f36ac-464d-4f32-bfec-6526fbe7640b-kube-api-access-s8dlv\") pod \"console-78bb99c546-zzlvc\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") " pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:23.850388 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:23.850293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:24.009847 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:24.009821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78bb99c546-zzlvc"]
Jul 06 15:30:24.012263 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:30:24.012231    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9f36ac_464d_4f32_bfec_6526fbe7640b.slice/crio-cb93e55d664d326aafc21a7cf9d205f348bbe6fe333b7370f00b99bb10de20fb WatchSource:0}: Error finding container cb93e55d664d326aafc21a7cf9d205f348bbe6fe333b7370f00b99bb10de20fb: Status 404 returned error can't find the container with id cb93e55d664d326aafc21a7cf9d205f348bbe6fe333b7370f00b99bb10de20fb
Jul 06 15:30:24.065874 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:24.065839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bb99c546-zzlvc" event={"ID":"3b9f36ac-464d-4f32-bfec-6526fbe7640b","Type":"ContainerStarted","Data":"cb93e55d664d326aafc21a7cf9d205f348bbe6fe333b7370f00b99bb10de20fb"}
Jul 06 15:30:25.071819 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:25.071775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bb99c546-zzlvc" event={"ID":"3b9f36ac-464d-4f32-bfec-6526fbe7640b","Type":"ContainerStarted","Data":"9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d"}
Jul 06 15:30:25.092615 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:25.092552    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78bb99c546-zzlvc" podStartSLOduration=2.092536091 podStartE2EDuration="2.092536091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:30:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:30:25.091260154 +0000 UTC m=+216.904122932" watchObservedRunningTime="2026-07-06 15:30:25.092536091 +0000 UTC m=+216.905398868"
Jul 06 15:30:33.851469 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:33.851424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:33.852006 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:33.851519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:33.856335 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:33.856306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:34.111944 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:34.111858    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:30:34.177791 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:34.177754    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcfdbbff8-2ldrh"]
Jul 06 15:30:59.201154 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.201115    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5fcfdbbff8-2ldrh" podUID="0191ee87-032b-4c01-b3e3-d7552d83bcad" containerName="console" containerID="cri-o://9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb" gracePeriod=15
Jul 06 15:30:59.460444 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.460363    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:30:59.541494 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-oauth-config\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.541723 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541508    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-oauth-serving-cert\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.541723 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541535    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-config\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.541723 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541666    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp4kl\" (UniqueName: \"kubernetes.io/projected/0191ee87-032b-4c01-b3e3-d7552d83bcad-kube-api-access-tp4kl\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.541897 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-trusted-ca-bundle\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.541897 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541770    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-serving-cert\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.541897 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.541797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-service-ca\") pod \"0191ee87-032b-4c01-b3e3-d7552d83bcad\" (UID: \"0191ee87-032b-4c01-b3e3-d7552d83bcad\") "
Jul 06 15:30:59.542045 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.542002    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:30:59.542045 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.542015    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-config" (OuterVolumeSpecName: "console-config") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:30:59.542142 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.542073    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:30:59.542142 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.542110    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-oauth-serving-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:59.542142 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.542132    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-config\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:59.542305 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.542287    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-service-ca" (OuterVolumeSpecName: "service-ca") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:30:59.544101 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.544070    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:59.544320 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.544291    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:30:59.544320 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.544305    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0191ee87-032b-4c01-b3e3-d7552d83bcad-kube-api-access-tp4kl" (OuterVolumeSpecName: "kube-api-access-tp4kl") pod "0191ee87-032b-4c01-b3e3-d7552d83bcad" (UID: "0191ee87-032b-4c01-b3e3-d7552d83bcad"). InnerVolumeSpecName "kube-api-access-tp4kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:30:59.643257 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.643206    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp4kl\" (UniqueName: \"kubernetes.io/projected/0191ee87-032b-4c01-b3e3-d7552d83bcad-kube-api-access-tp4kl\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:59.643257 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.643242    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-trusted-ca-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:59.643257 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.643254    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-serving-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:59.643257 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.643262    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0191ee87-032b-4c01-b3e3-d7552d83bcad-service-ca\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:30:59.643257 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:30:59.643271    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0191ee87-032b-4c01-b3e3-d7552d83bcad-console-oauth-config\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:31:00.214795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.214752    2580 generic.go:356] "Generic (PLEG): container finished" podID="0191ee87-032b-4c01-b3e3-d7552d83bcad" containerID="9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb" exitCode=0
Jul 06 15:31:00.215194 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.214824    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcfdbbff8-2ldrh"
Jul 06 15:31:00.215194 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.214838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfdbbff8-2ldrh" event={"ID":"0191ee87-032b-4c01-b3e3-d7552d83bcad","Type":"ContainerDied","Data":"9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb"}
Jul 06 15:31:00.215194 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.214897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcfdbbff8-2ldrh" event={"ID":"0191ee87-032b-4c01-b3e3-d7552d83bcad","Type":"ContainerDied","Data":"4dd62d26518c5de9cf2adfb161936eb04ea5afd25e177cea5db743e0cc574c9b"}
Jul 06 15:31:00.215194 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.214919    2580 scope.go:117] "RemoveContainer" containerID="9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb"
Jul 06 15:31:00.223983 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.223962    2580 scope.go:117] "RemoveContainer" containerID="9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb"
Jul 06 15:31:00.224317 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:31:00.224296    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb\": container with ID starting with 9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb not found: ID does not exist" containerID="9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb"
Jul 06 15:31:00.224379 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.224334    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb"} err="failed to get container status \"9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb\": rpc error: code = NotFound desc = could not find container \"9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb\": container with ID starting with 9f5c3fbc0d1c9cfacc7c6307a04ce15f550b60b142747c251e3fdc6314942dcb not found: ID does not exist"
Jul 06 15:31:00.232934 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.232904    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcfdbbff8-2ldrh"]
Jul 06 15:31:00.237785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:00.237752    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fcfdbbff8-2ldrh"]
Jul 06 15:31:02.021057 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:02.021025    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0191ee87-032b-4c01-b3e3-d7552d83bcad" path="/var/lib/kubelet/pods/0191ee87-032b-4c01-b3e3-d7552d83bcad/volumes"
Jul 06 15:31:28.446377 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.446339    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"]
Jul 06 15:31:28.446910 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.446741    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0191ee87-032b-4c01-b3e3-d7552d83bcad" containerName="console"
Jul 06 15:31:28.446910 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.446755    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0191ee87-032b-4c01-b3e3-d7552d83bcad" containerName="console"
Jul 06 15:31:28.446910 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.446817    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0191ee87-032b-4c01-b3e3-d7552d83bcad" containerName="console"
Jul 06 15:31:28.450204 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.450187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.453541 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.453515    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j9f9c\""
Jul 06 15:31:28.454240 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.454221    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 15:31:28.454410 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.454390    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 15:31:28.465822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.465792    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"]
Jul 06 15:31:28.589983 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.589934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w26hm\" (UniqueName: \"kubernetes.io/projected/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-kube-api-access-w26hm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.589983 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.589985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.590241 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.590017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.691136 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.691094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.691136 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.691131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.691343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.691211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w26hm\" (UniqueName: \"kubernetes.io/projected/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-kube-api-access-w26hm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.691510 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.691492    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.691564 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.691544    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.700481 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.700413    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w26hm\" (UniqueName: \"kubernetes.io/projected/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-kube-api-access-w26hm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.759320 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.759272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:28.901177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:28.901138    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"]
Jul 06 15:31:28.904314 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:31:28.904284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9562c4_7761_4c10_a6ed_09e8dfbc8cb3.slice/crio-607d207b605d363bb09484b949b63d1ab9764078922a77d7f1d8c74d190827ca WatchSource:0}: Error finding container 607d207b605d363bb09484b949b63d1ab9764078922a77d7f1d8c74d190827ca: Status 404 returned error can't find the container with id 607d207b605d363bb09484b949b63d1ab9764078922a77d7f1d8c74d190827ca
Jul 06 15:31:29.334438 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:29.334402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw" event={"ID":"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3","Type":"ContainerStarted","Data":"607d207b605d363bb09484b949b63d1ab9764078922a77d7f1d8c74d190827ca"}
Jul 06 15:31:37.371513 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:37.371473    2580 generic.go:356] "Generic (PLEG): container finished" podID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerID="ed1731034fc04469a3e37260d74114cc76f6af8ba1f5815f3c495bd76047b237" exitCode=0
Jul 06 15:31:37.371975 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:37.371558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw" event={"ID":"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3","Type":"ContainerDied","Data":"ed1731034fc04469a3e37260d74114cc76f6af8ba1f5815f3c495bd76047b237"}
Jul 06 15:31:40.388609 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:40.388575    2580 generic.go:356] "Generic (PLEG): container finished" podID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerID="3cb91cc8fb62c7251f9e71639ad91a786da20145b58339139a91c9b92d523d9e" exitCode=0
Jul 06 15:31:40.389004 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:40.388642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw" event={"ID":"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3","Type":"ContainerDied","Data":"3cb91cc8fb62c7251f9e71639ad91a786da20145b58339139a91c9b92d523d9e"}
Jul 06 15:31:50.061251 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:50.061219    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:31:50.061251 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:50.061235    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:31:50.065369 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:50.065343    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 15:31:50.435256 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:50.435218    2580 generic.go:356] "Generic (PLEG): container finished" podID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerID="3a78d4a0130cda6727594e772abde35ea4fa509b9959e381a1c0d3c23738b201" exitCode=0
Jul 06 15:31:50.435416 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:50.435273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw" event={"ID":"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3","Type":"ContainerDied","Data":"3a78d4a0130cda6727594e772abde35ea4fa509b9959e381a1c0d3c23738b201"}
Jul 06 15:31:51.624025 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.623987    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:51.676536 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.676497    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w26hm\" (UniqueName: \"kubernetes.io/projected/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-kube-api-access-w26hm\") pod \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") "
Jul 06 15:31:51.676536 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.676556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-bundle\") pod \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") "
Jul 06 15:31:51.676780 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.676582    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-util\") pod \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\" (UID: \"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3\") "
Jul 06 15:31:51.677280 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.677244    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-bundle" (OuterVolumeSpecName: "bundle") pod "7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" (UID: "7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:31:51.678861 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.678834    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-kube-api-access-w26hm" (OuterVolumeSpecName: "kube-api-access-w26hm") pod "7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" (UID: "7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3"). InnerVolumeSpecName "kube-api-access-w26hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:31:51.680630 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.680607    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-util" (OuterVolumeSpecName: "util") pod "7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" (UID: "7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:31:51.778220 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.778127    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w26hm\" (UniqueName: \"kubernetes.io/projected/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-kube-api-access-w26hm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:31:51.778220 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.778199    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:31:51.778220 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:51.778213    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:31:52.445121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:52.445091    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw"
Jul 06 15:31:52.445121 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:52.445106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675402bknw" event={"ID":"7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3","Type":"ContainerDied","Data":"607d207b605d363bb09484b949b63d1ab9764078922a77d7f1d8c74d190827ca"}
Jul 06 15:31:52.445360 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:52.445163    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="607d207b605d363bb09484b949b63d1ab9764078922a77d7f1d8c74d190827ca"
Jul 06 15:31:56.548477 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548441    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"]
Jul 06 15:31:56.548884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548805    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="util"
Jul 06 15:31:56.548884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548817    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="util"
Jul 06 15:31:56.548884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548827    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="extract"
Jul 06 15:31:56.548884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548833    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="extract"
Jul 06 15:31:56.548884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548849    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="pull"
Jul 06 15:31:56.548884 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548855    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="pull"
Jul 06 15:31:56.549075 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.548904    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9562c4-7761-4c10-a6ed-09e8dfbc8cb3" containerName="extract"
Jul 06 15:31:56.551014 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.550996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.553522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.553486    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 06 15:31:56.553735 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.553650    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 06 15:31:56.553836 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.553814    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zv2bb\""
Jul 06 15:31:56.571449 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.571409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"]
Jul 06 15:31:56.621409 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.621367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjqdv\" (UID: \"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.621596 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.621426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6wkd\" (UniqueName: \"kubernetes.io/projected/bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b-kube-api-access-c6wkd\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjqdv\" (UID: \"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.722494 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.722452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6wkd\" (UniqueName: \"kubernetes.io/projected/bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b-kube-api-access-c6wkd\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjqdv\" (UID: \"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.722633 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.722551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjqdv\" (UID: \"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.722969 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.722951    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjqdv\" (UID: \"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.735143 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.735108    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6wkd\" (UniqueName: \"kubernetes.io/projected/bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b-kube-api-access-c6wkd\") pod \"cert-manager-operator-controller-manager-55c444bf86-zjqdv\" (UID: \"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:56.860043 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:56.859911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"
Jul 06 15:31:57.020509 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:57.020473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv"]
Jul 06 15:31:57.024495 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:31:57.024460    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5bc1ba_5bac_410a_b5d6_5f0ed003ae7b.slice/crio-90557cd91becf80e23f46bf62a3e9e1f69f79e2294955cb474b2d464f64d13d1 WatchSource:0}: Error finding container 90557cd91becf80e23f46bf62a3e9e1f69f79e2294955cb474b2d464f64d13d1: Status 404 returned error can't find the container with id 90557cd91becf80e23f46bf62a3e9e1f69f79e2294955cb474b2d464f64d13d1
Jul 06 15:31:57.028044 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:57.028025    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 15:31:57.473260 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:57.473217    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv" event={"ID":"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b","Type":"ContainerStarted","Data":"90557cd91becf80e23f46bf62a3e9e1f69f79e2294955cb474b2d464f64d13d1"}
Jul 06 15:31:59.483631 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:59.483597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv" event={"ID":"bb5bc1ba-5bac-410a-b5d6-5f0ed003ae7b","Type":"ContainerStarted","Data":"d45db5b4cc7b366f87729d9b91150c38c8f4dd18db95506cd853c757bb1cb95d"}
Jul 06 15:31:59.512509 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:31:59.512450    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zjqdv" podStartSLOduration=2.048919208 podStartE2EDuration="3.512435189s" totalImagesPullingTime="1.463515981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:31:59.510694717 +0000 UTC m=+311.323557496" watchObservedRunningTime="2026-07-06 15:31:59.512435189 +0000 UTC m=+311.325297966"
Jul 06 15:32:00.189785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.189743    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"]
Jul 06 15:32:00.192400 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.192381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.194686 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.194656    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j9f9c\""
Jul 06 15:32:00.194686 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.194661    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 15:32:00.195392 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.195369    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 15:32:00.217859 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.217818    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"]
Jul 06 15:32:00.254246 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.254207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cknmk\" (UniqueName: \"kubernetes.io/projected/a4de6685-f6e1-49b5-a86f-94954d919b78-kube-api-access-cknmk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.254438 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.254283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.254438 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.254326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.355154 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.355117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.355361 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.355176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.355361 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.355227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cknmk\" (UniqueName: \"kubernetes.io/projected/a4de6685-f6e1-49b5-a86f-94954d919b78-kube-api-access-cknmk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.355528 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.355504    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.355582 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.355535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.364499 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.364458    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cknmk\" (UniqueName: \"kubernetes.io/projected/a4de6685-f6e1-49b5-a86f-94954d919b78-kube-api-access-cknmk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.502313 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.502223    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:00.650895 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:00.650867    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"]
Jul 06 15:32:00.653796 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:32:00.653758    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4de6685_f6e1_49b5_a86f_94954d919b78.slice/crio-e3ad58488c1b6444f0b3e240dd8d89dd18a261bbaa52c4533c38c92cbd889443 WatchSource:0}: Error finding container e3ad58488c1b6444f0b3e240dd8d89dd18a261bbaa52c4533c38c92cbd889443: Status 404 returned error can't find the container with id e3ad58488c1b6444f0b3e240dd8d89dd18a261bbaa52c4533c38c92cbd889443
Jul 06 15:32:01.494563 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:01.494527    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerID="4515cd31b41514790df31b4407350e920c8d48d3d456042ca6037b8c36582eef" exitCode=0
Jul 06 15:32:01.494822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:01.494591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw" event={"ID":"a4de6685-f6e1-49b5-a86f-94954d919b78","Type":"ContainerDied","Data":"4515cd31b41514790df31b4407350e920c8d48d3d456042ca6037b8c36582eef"}
Jul 06 15:32:01.494822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:01.494626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw" event={"ID":"a4de6685-f6e1-49b5-a86f-94954d919b78","Type":"ContainerStarted","Data":"e3ad58488c1b6444f0b3e240dd8d89dd18a261bbaa52c4533c38c92cbd889443"}
Jul 06 15:32:03.104309 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.104266    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8tjkv"]
Jul 06 15:32:03.110446 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.110417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.112893 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.112839    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 15:32:03.113049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.112901    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 15:32:03.113540 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.113521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-j99gr\""
Jul 06 15:32:03.120625 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.120595    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8tjkv"]
Jul 06 15:32:03.128396 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.128363    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"]
Jul 06 15:32:03.131263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.131239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.133567 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.133537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-js5bn\""
Jul 06 15:32:03.145726 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.145678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"]
Jul 06 15:32:03.180844 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.180803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39f18fae-572c-4adf-a478-b613ea3c8148-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8tjkv\" (UID: \"39f18fae-572c-4adf-a478-b613ea3c8148\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.181040 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.180855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbq7z\" (UniqueName: \"kubernetes.io/projected/537a6080-384b-4ed0-b4cd-38bb7978f17f-kube-api-access-dbq7z\") pod \"cert-manager-cainjector-559cdd8c87-bq5qf\" (UID: \"537a6080-384b-4ed0-b4cd-38bb7978f17f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.181040 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.180878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/537a6080-384b-4ed0-b4cd-38bb7978f17f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bq5qf\" (UID: \"537a6080-384b-4ed0-b4cd-38bb7978f17f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.181040 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.180963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjl86\" (UniqueName: \"kubernetes.io/projected/39f18fae-572c-4adf-a478-b613ea3c8148-kube-api-access-mjl86\") pod \"cert-manager-webhook-6486ffd997-8tjkv\" (UID: \"39f18fae-572c-4adf-a478-b613ea3c8148\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.281663 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.281627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39f18fae-572c-4adf-a478-b613ea3c8148-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8tjkv\" (UID: \"39f18fae-572c-4adf-a478-b613ea3c8148\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.281880 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.281709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbq7z\" (UniqueName: \"kubernetes.io/projected/537a6080-384b-4ed0-b4cd-38bb7978f17f-kube-api-access-dbq7z\") pod \"cert-manager-cainjector-559cdd8c87-bq5qf\" (UID: \"537a6080-384b-4ed0-b4cd-38bb7978f17f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.281880 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.281739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/537a6080-384b-4ed0-b4cd-38bb7978f17f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bq5qf\" (UID: \"537a6080-384b-4ed0-b4cd-38bb7978f17f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.281880 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.281776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjl86\" (UniqueName: \"kubernetes.io/projected/39f18fae-572c-4adf-a478-b613ea3c8148-kube-api-access-mjl86\") pod \"cert-manager-webhook-6486ffd997-8tjkv\" (UID: \"39f18fae-572c-4adf-a478-b613ea3c8148\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.290782 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.290694    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39f18fae-572c-4adf-a478-b613ea3c8148-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8tjkv\" (UID: \"39f18fae-572c-4adf-a478-b613ea3c8148\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.291044 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.291014    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjl86\" (UniqueName: \"kubernetes.io/projected/39f18fae-572c-4adf-a478-b613ea3c8148-kube-api-access-mjl86\") pod \"cert-manager-webhook-6486ffd997-8tjkv\" (UID: \"39f18fae-572c-4adf-a478-b613ea3c8148\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.291393 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.291368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/537a6080-384b-4ed0-b4cd-38bb7978f17f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bq5qf\" (UID: \"537a6080-384b-4ed0-b4cd-38bb7978f17f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.291557 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.291540    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbq7z\" (UniqueName: \"kubernetes.io/projected/537a6080-384b-4ed0-b4cd-38bb7978f17f-kube-api-access-dbq7z\") pod \"cert-manager-cainjector-559cdd8c87-bq5qf\" (UID: \"537a6080-384b-4ed0-b4cd-38bb7978f17f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.431244 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.431206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:03.442056 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.442023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"
Jul 06 15:32:03.644422 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.644256    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf"]
Jul 06 15:32:03.647843 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:32:03.647815    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod537a6080_384b_4ed0_b4cd_38bb7978f17f.slice/crio-90811075d7306a119abf1e6cd06859e48b2ab5d094b3bc2216500179c59a7b76 WatchSource:0}: Error finding container 90811075d7306a119abf1e6cd06859e48b2ab5d094b3bc2216500179c59a7b76: Status 404 returned error can't find the container with id 90811075d7306a119abf1e6cd06859e48b2ab5d094b3bc2216500179c59a7b76
Jul 06 15:32:03.661779 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:03.661739    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8tjkv"]
Jul 06 15:32:03.682694 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:32:03.682611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39f18fae_572c_4adf_a478_b613ea3c8148.slice/crio-5dec226e3e845090b024d33579cf23abfcbdf45a0520e3b3a1cc85eab29dcb88 WatchSource:0}: Error finding container 5dec226e3e845090b024d33579cf23abfcbdf45a0520e3b3a1cc85eab29dcb88: Status 404 returned error can't find the container with id 5dec226e3e845090b024d33579cf23abfcbdf45a0520e3b3a1cc85eab29dcb88
Jul 06 15:32:04.512405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:04.512365    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerID="1a414b622ba643fe9b394693215313e4d2afa138e9cff387e12557cdb52cb768" exitCode=0
Jul 06 15:32:04.512905 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:04.512466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw" event={"ID":"a4de6685-f6e1-49b5-a86f-94954d919b78","Type":"ContainerDied","Data":"1a414b622ba643fe9b394693215313e4d2afa138e9cff387e12557cdb52cb768"}
Jul 06 15:32:04.513719 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:04.513674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf" event={"ID":"537a6080-384b-4ed0-b4cd-38bb7978f17f","Type":"ContainerStarted","Data":"90811075d7306a119abf1e6cd06859e48b2ab5d094b3bc2216500179c59a7b76"}
Jul 06 15:32:04.514792 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:04.514770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv" event={"ID":"39f18fae-572c-4adf-a478-b613ea3c8148","Type":"ContainerStarted","Data":"5dec226e3e845090b024d33579cf23abfcbdf45a0520e3b3a1cc85eab29dcb88"}
Jul 06 15:32:05.523631 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:05.523586    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerID="bd02cc4c4331e3f731ca06d3c9be94528e65227f08c602e882415d2443678b66" exitCode=0
Jul 06 15:32:05.524093 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:05.523668    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw" event={"ID":"a4de6685-f6e1-49b5-a86f-94954d919b78","Type":"ContainerDied","Data":"bd02cc4c4331e3f731ca06d3c9be94528e65227f08c602e882415d2443678b66"}
Jul 06 15:32:07.010423 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.010393    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:07.114957 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.114921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cknmk\" (UniqueName: \"kubernetes.io/projected/a4de6685-f6e1-49b5-a86f-94954d919b78-kube-api-access-cknmk\") pod \"a4de6685-f6e1-49b5-a86f-94954d919b78\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") "
Jul 06 15:32:07.115149 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.114966    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-util\") pod \"a4de6685-f6e1-49b5-a86f-94954d919b78\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") "
Jul 06 15:32:07.115149 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.115044    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-bundle\") pod \"a4de6685-f6e1-49b5-a86f-94954d919b78\" (UID: \"a4de6685-f6e1-49b5-a86f-94954d919b78\") "
Jul 06 15:32:07.115415 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.115390    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-bundle" (OuterVolumeSpecName: "bundle") pod "a4de6685-f6e1-49b5-a86f-94954d919b78" (UID: "a4de6685-f6e1-49b5-a86f-94954d919b78"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:32:07.117181 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.117158    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4de6685-f6e1-49b5-a86f-94954d919b78-kube-api-access-cknmk" (OuterVolumeSpecName: "kube-api-access-cknmk") pod "a4de6685-f6e1-49b5-a86f-94954d919b78" (UID: "a4de6685-f6e1-49b5-a86f-94954d919b78"). InnerVolumeSpecName "kube-api-access-cknmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:32:07.119465 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.119442    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-util" (OuterVolumeSpecName: "util") pod "a4de6685-f6e1-49b5-a86f-94954d919b78" (UID: "a4de6685-f6e1-49b5-a86f-94954d919b78"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:32:07.216389 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.216344    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:32:07.216389 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.216377    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cknmk\" (UniqueName: \"kubernetes.io/projected/a4de6685-f6e1-49b5-a86f-94954d919b78-kube-api-access-cknmk\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:32:07.216389 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.216387    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4de6685-f6e1-49b5-a86f-94954d919b78-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:32:07.535062 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.535016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf" event={"ID":"537a6080-384b-4ed0-b4cd-38bb7978f17f","Type":"ContainerStarted","Data":"3d61e9e8d5a6655631a983fdbbc7dbf331e035e7ab2646ea6d7b1e6b98cda677"}
Jul 06 15:32:07.536843 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.536817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw" event={"ID":"a4de6685-f6e1-49b5-a86f-94954d919b78","Type":"ContainerDied","Data":"e3ad58488c1b6444f0b3e240dd8d89dd18a261bbaa52c4533c38c92cbd889443"}
Jul 06 15:32:07.536986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.536847    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb52nw"
Jul 06 15:32:07.536986 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.536858    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3ad58488c1b6444f0b3e240dd8d89dd18a261bbaa52c4533c38c92cbd889443"
Jul 06 15:32:07.560396 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:07.560325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bq5qf" podStartSLOduration=1.148102904 podStartE2EDuration="4.560302393s" totalImagesPullingTime="3.412199489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:32:07.559107005 +0000 UTC m=+319.371969784" watchObservedRunningTime="2026-07-06 15:32:07.560302393 +0000 UTC m=+319.373165172"
Jul 06 15:32:16.583668 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:16.583580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv" event={"ID":"39f18fae-572c-4adf-a478-b613ea3c8148","Type":"ContainerStarted","Data":"64a31abdb16d70cf785879b32dd3053a092a747a4d24c4b910bb9fe191eb200c"}
Jul 06 15:32:16.584089 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:16.583733    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:16.610648 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:16.610569    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv" podStartSLOduration=1.13903853 podStartE2EDuration="13.610551995s" totalImagesPullingTime="12.471513465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:32:16.609636114 +0000 UTC m=+328.422498901" watchObservedRunningTime="2026-07-06 15:32:16.610551995 +0000 UTC m=+328.423414772"
Jul 06 15:32:22.590873 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:22.590835    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8tjkv"
Jul 06 15:32:31.263740 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.263687    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"]
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264042    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="extract"
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264053    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="extract"
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264072    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="util"
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264077    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="util"
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264086    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="pull"
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264092    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="pull"
Jul 06 15:32:31.264177 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.264147    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4de6685-f6e1-49b5-a86f-94954d919b78" containerName="extract"
Jul 06 15:32:31.266443 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.266426    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.268927 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.268900    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 15:32:31.269063 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.268978    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 15:32:31.269626 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.269605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j9f9c\""
Jul 06 15:32:31.276899 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.276868    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"]
Jul 06 15:32:31.436861 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.436815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.437036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.436927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.437036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.436974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84mmw\" (UniqueName: \"kubernetes.io/projected/7abfa76d-50ac-4259-9180-4ae9ce396808-kube-api-access-84mmw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.537640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.537551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.537640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.537611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84mmw\" (UniqueName: \"kubernetes.io/projected/7abfa76d-50ac-4259-9180-4ae9ce396808-kube-api-access-84mmw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.537861 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.537651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.537995 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.537979    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.538036 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.537994    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.546790 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.546760    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84mmw\" (UniqueName: \"kubernetes.io/projected/7abfa76d-50ac-4259-9180-4ae9ce396808-kube-api-access-84mmw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.576685 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.576644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:31.758862 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:31.758834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"]
Jul 06 15:32:31.761615 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:32:31.761579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7abfa76d_50ac_4259_9180_4ae9ce396808.slice/crio-77966eb14db7256fc273951e9c487439c606976d7f397f01b691dad88db5a5e9 WatchSource:0}: Error finding container 77966eb14db7256fc273951e9c487439c606976d7f397f01b691dad88db5a5e9: Status 404 returned error can't find the container with id 77966eb14db7256fc273951e9c487439c606976d7f397f01b691dad88db5a5e9
Jul 06 15:32:32.663594 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:32.663560    2580 generic.go:356] "Generic (PLEG): container finished" podID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerID="6c61754eb1d2d6b362ca060fb455cfff8c2d0dc7c6bf98fda131249e5112bd96" exitCode=0
Jul 06 15:32:32.664010 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:32.663615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk" event={"ID":"7abfa76d-50ac-4259-9180-4ae9ce396808","Type":"ContainerDied","Data":"6c61754eb1d2d6b362ca060fb455cfff8c2d0dc7c6bf98fda131249e5112bd96"}
Jul 06 15:32:32.664010 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:32.663655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk" event={"ID":"7abfa76d-50ac-4259-9180-4ae9ce396808","Type":"ContainerStarted","Data":"77966eb14db7256fc273951e9c487439c606976d7f397f01b691dad88db5a5e9"}
Jul 06 15:32:34.674763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:34.674725    2580 generic.go:356] "Generic (PLEG): container finished" podID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerID="15396ece52736cc9ff62bcd48c243b34457f57d050ef2d45b23b2cb558c4b1b2" exitCode=0
Jul 06 15:32:34.675235 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:34.674799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk" event={"ID":"7abfa76d-50ac-4259-9180-4ae9ce396808","Type":"ContainerDied","Data":"15396ece52736cc9ff62bcd48c243b34457f57d050ef2d45b23b2cb558c4b1b2"}
Jul 06 15:32:35.682111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:35.682076    2580 generic.go:356] "Generic (PLEG): container finished" podID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerID="ef6ce4528b95f13a9d8784879c1c67c52fcb5aac703191817160cbf531fbd40a" exitCode=0
Jul 06 15:32:35.682520 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:35.682160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk" event={"ID":"7abfa76d-50ac-4259-9180-4ae9ce396808","Type":"ContainerDied","Data":"ef6ce4528b95f13a9d8784879c1c67c52fcb5aac703191817160cbf531fbd40a"}
Jul 06 15:32:36.821713 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.821671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:36.885293 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.885260    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-util\") pod \"7abfa76d-50ac-4259-9180-4ae9ce396808\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") "
Jul 06 15:32:36.885468 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.885330    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84mmw\" (UniqueName: \"kubernetes.io/projected/7abfa76d-50ac-4259-9180-4ae9ce396808-kube-api-access-84mmw\") pod \"7abfa76d-50ac-4259-9180-4ae9ce396808\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") "
Jul 06 15:32:36.885468 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.885385    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-bundle\") pod \"7abfa76d-50ac-4259-9180-4ae9ce396808\" (UID: \"7abfa76d-50ac-4259-9180-4ae9ce396808\") "
Jul 06 15:32:36.886122 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.886093    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-bundle" (OuterVolumeSpecName: "bundle") pod "7abfa76d-50ac-4259-9180-4ae9ce396808" (UID: "7abfa76d-50ac-4259-9180-4ae9ce396808"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:32:36.887654 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.887620    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7abfa76d-50ac-4259-9180-4ae9ce396808-kube-api-access-84mmw" (OuterVolumeSpecName: "kube-api-access-84mmw") pod "7abfa76d-50ac-4259-9180-4ae9ce396808" (UID: "7abfa76d-50ac-4259-9180-4ae9ce396808"). InnerVolumeSpecName "kube-api-access-84mmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:32:36.891286 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.891256    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-util" (OuterVolumeSpecName: "util") pod "7abfa76d-50ac-4259-9180-4ae9ce396808" (UID: "7abfa76d-50ac-4259-9180-4ae9ce396808"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:32:36.986205 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.986114    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84mmw\" (UniqueName: \"kubernetes.io/projected/7abfa76d-50ac-4259-9180-4ae9ce396808-kube-api-access-84mmw\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:32:36.986205 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.986144    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:32:36.986205 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:36.986153    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7abfa76d-50ac-4259-9180-4ae9ce396808-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:32:37.693787 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:37.693750    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk" event={"ID":"7abfa76d-50ac-4259-9180-4ae9ce396808","Type":"ContainerDied","Data":"77966eb14db7256fc273951e9c487439c606976d7f397f01b691dad88db5a5e9"}
Jul 06 15:32:37.693971 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:37.693808    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77966eb14db7256fc273951e9c487439c606976d7f397f01b691dad88db5a5e9"
Jul 06 15:32:37.693971 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:37.693763    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwtfrk"
Jul 06 15:32:48.473545 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473460    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"]
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473865    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="extract"
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473877    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="extract"
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473901    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="pull"
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473908    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="pull"
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473920    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="util"
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473925    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="util"
Jul 06 15:32:48.474017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.473987    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7abfa76d-50ac-4259-9180-4ae9ce396808" containerName="extract"
Jul 06 15:32:48.476273 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.476249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.481447 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.481424    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 15:32:48.483674 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.483649    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 15:32:48.485017 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.484997    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d8sts\""
Jul 06 15:32:48.485124 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.485108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 15:32:48.485872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.485857    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 15:32:48.512538 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.512495    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"]
Jul 06 15:32:48.574900 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.574849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bacaebb-2ceb-4b70-a61d-2708192816a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.575108 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.574913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bacaebb-2ceb-4b70-a61d-2708192816a6-manifests\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.575108 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.574950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tblkq\" (UniqueName: \"kubernetes.io/projected/6bacaebb-2ceb-4b70-a61d-2708192816a6-kube-api-access-tblkq\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.575108 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.574975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bacaebb-2ceb-4b70-a61d-2708192816a6-tmp\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.575108 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.575025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bacaebb-2ceb-4b70-a61d-2708192816a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.676438 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.676393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bacaebb-2ceb-4b70-a61d-2708192816a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.676642 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.676457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bacaebb-2ceb-4b70-a61d-2708192816a6-manifests\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.676642 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.676495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tblkq\" (UniqueName: \"kubernetes.io/projected/6bacaebb-2ceb-4b70-a61d-2708192816a6-kube-api-access-tblkq\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.676642 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.676525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bacaebb-2ceb-4b70-a61d-2708192816a6-tmp\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.676642 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.676566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bacaebb-2ceb-4b70-a61d-2708192816a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.677108 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.677082    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bacaebb-2ceb-4b70-a61d-2708192816a6-manifests\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.679077 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.679055    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bacaebb-2ceb-4b70-a61d-2708192816a6-tmp\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.680360 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.680337    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 15:32:48.689851 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.689813    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bacaebb-2ceb-4b70-a61d-2708192816a6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.689851 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.689849    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bacaebb-2ceb-4b70-a61d-2708192816a6-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.705658 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.705623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 15:32:48.715513 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.715488    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 15:32:48.726116 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.726045    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tblkq\" (UniqueName: \"kubernetes.io/projected/6bacaebb-2ceb-4b70-a61d-2708192816a6-kube-api-access-tblkq\") pod \"opendatahub-operator-controller-manager-5bdc85857f-c7nbn\" (UID: \"6bacaebb-2ceb-4b70-a61d-2708192816a6\") " pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.789844 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.789807    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d8sts\""
Jul 06 15:32:48.798234 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.798200    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:48.962066 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:48.962043    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"]
Jul 06 15:32:48.964859 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:32:48.964832    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bacaebb_2ceb_4b70_a61d_2708192816a6.slice/crio-bf871f90e1c953ab9aa8aa44b74d0aba59289d34e23d60543c7b00b0947f710b WatchSource:0}: Error finding container bf871f90e1c953ab9aa8aa44b74d0aba59289d34e23d60543c7b00b0947f710b: Status 404 returned error can't find the container with id bf871f90e1c953ab9aa8aa44b74d0aba59289d34e23d60543c7b00b0947f710b
Jul 06 15:32:49.756032 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:49.755995    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn" event={"ID":"6bacaebb-2ceb-4b70-a61d-2708192816a6","Type":"ContainerStarted","Data":"bf871f90e1c953ab9aa8aa44b74d0aba59289d34e23d60543c7b00b0947f710b"}
Jul 06 15:32:52.774012 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:52.773969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn" event={"ID":"6bacaebb-2ceb-4b70-a61d-2708192816a6","Type":"ContainerStarted","Data":"67f5b2d0b4af4f403f2b97eaa3e71901508babbf69bbaf49ff0fdd5e118f1ca2"}
Jul 06 15:32:53.780908 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:53.780811    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bacaebb-2ceb-4b70-a61d-2708192816a6" containerID="67f5b2d0b4af4f403f2b97eaa3e71901508babbf69bbaf49ff0fdd5e118f1ca2" exitCode=0
Jul 06 15:32:53.781349 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:53.780900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn" event={"ID":"6bacaebb-2ceb-4b70-a61d-2708192816a6","Type":"ContainerDied","Data":"67f5b2d0b4af4f403f2b97eaa3e71901508babbf69bbaf49ff0fdd5e118f1ca2"}
Jul 06 15:32:54.788332 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:54.788295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn" event={"ID":"6bacaebb-2ceb-4b70-a61d-2708192816a6","Type":"ContainerStarted","Data":"1e1a4a12d70049f44377af6d1921aa1cdb3262049dc0ef0eba256d393bb81104"}
Jul 06 15:32:54.788790 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:54.788434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:32:54.856556 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:32:54.856496    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn" podStartSLOduration=2.00946978 podStartE2EDuration="6.856478136s" totalImagesPullingTime="3.706241411s" totalInitContainerRuntime="1.140766945s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:32:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:32:54.854016967 +0000 UTC m=+366.666879770" watchObservedRunningTime="2026-07-06 15:32:54.856478136 +0000 UTC m=+366.669340914"
Jul 06 15:33:05.796245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:05.796213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bdc85857f-c7nbn"
Jul 06 15:33:09.395311 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.395271    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tqwl7"]
Jul 06 15:33:09.406004 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.405960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:09.413763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.413572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-q447q\""
Jul 06 15:33:09.413763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.413573    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 15:33:09.415512 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.415485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tqwl7"]
Jul 06 15:33:09.562913 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.562867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-cert\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:09.563088 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.562922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7xxj\" (UniqueName: \"kubernetes.io/projected/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-kube-api-access-l7xxj\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:09.664366 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.664265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-cert\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:09.664366 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.664317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7xxj\" (UniqueName: \"kubernetes.io/projected/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-kube-api-access-l7xxj\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:09.664579 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:33:09.664421    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 15:33:09.664579 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:33:09.664490    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-cert podName:1a3665d2-dfb6-4704-847a-4b9ee5a376a8 nodeName:}" failed. No retries permitted until 2026-07-06 15:33:10.164472586 +0000 UTC m=+381.977335342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-cert") pod "odh-model-controller-7998c67dc4-tqwl7" (UID: "1a3665d2-dfb6-4704-847a-4b9ee5a376a8") : secret "odh-model-controller-webhook-cert" not found
Jul 06 15:33:09.687810 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:09.687776    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7xxj\" (UniqueName: \"kubernetes.io/projected/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-kube-api-access-l7xxj\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:10.107639 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.107550    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8frj8"]
Jul 06 15:33:10.110020 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.110000    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.115399 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.115375    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 15:33:10.126003 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.125968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8frj8"]
Jul 06 15:33:10.168973 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.168930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-cert\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:10.171414 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.171387    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a3665d2-dfb6-4704-847a-4b9ee5a376a8-cert\") pod \"odh-model-controller-7998c67dc4-tqwl7\" (UID: \"1a3665d2-dfb6-4704-847a-4b9ee5a376a8\") " pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:10.269788 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.269750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz65l\" (UniqueName: \"kubernetes.io/projected/8788900f-ff02-4349-96d0-b22184666734-kube-api-access-xz65l\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.269963 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.269797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8788900f-ff02-4349-96d0-b22184666734-tls-certs\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.325980 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.325925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:10.371171 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.371134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz65l\" (UniqueName: \"kubernetes.io/projected/8788900f-ff02-4349-96d0-b22184666734-kube-api-access-xz65l\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.371171 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.371183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8788900f-ff02-4349-96d0-b22184666734-tls-certs\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.371455 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:33:10.371300    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 06 15:33:10.371455 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:33:10.371357    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8788900f-ff02-4349-96d0-b22184666734-tls-certs podName:8788900f-ff02-4349-96d0-b22184666734 nodeName:}" failed. No retries permitted until 2026-07-06 15:33:10.871339453 +0000 UTC m=+382.684202209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8788900f-ff02-4349-96d0-b22184666734-tls-certs") pod "model-serving-api-54d89bd5fc-8frj8" (UID: "8788900f-ff02-4349-96d0-b22184666734") : secret "model-serving-api-tls" not found
Jul 06 15:33:10.383890 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.383861    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz65l\" (UniqueName: \"kubernetes.io/projected/8788900f-ff02-4349-96d0-b22184666734-kube-api-access-xz65l\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.475003 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.474970    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tqwl7"]
Jul 06 15:33:10.478126 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:33:10.478097    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3665d2_dfb6_4704_847a_4b9ee5a376a8.slice/crio-38aaf902a68396e050f0230269765d04d8e583c919e9f53d2de99e1e104ec3aa WatchSource:0}: Error finding container 38aaf902a68396e050f0230269765d04d8e583c919e9f53d2de99e1e104ec3aa: Status 404 returned error can't find the container with id 38aaf902a68396e050f0230269765d04d8e583c919e9f53d2de99e1e104ec3aa
Jul 06 15:33:10.866806 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.866684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7" event={"ID":"1a3665d2-dfb6-4704-847a-4b9ee5a376a8","Type":"ContainerStarted","Data":"38aaf902a68396e050f0230269765d04d8e583c919e9f53d2de99e1e104ec3aa"}
Jul 06 15:33:10.877326 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.877293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8788900f-ff02-4349-96d0-b22184666734-tls-certs\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:10.879906 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:10.879879    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8788900f-ff02-4349-96d0-b22184666734-tls-certs\") pod \"model-serving-api-54d89bd5fc-8frj8\" (UID: \"8788900f-ff02-4349-96d0-b22184666734\") " pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:11.021195 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:11.021155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:11.163419 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:11.163381    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8frj8"]
Jul 06 15:33:11.166932 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:33:11.166894    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8788900f_ff02_4349_96d0_b22184666734.slice/crio-01d05415d0b8442725a7ec93e68ca9f91da4c5d9af1a572e9750a17e96f708ab WatchSource:0}: Error finding container 01d05415d0b8442725a7ec93e68ca9f91da4c5d9af1a572e9750a17e96f708ab: Status 404 returned error can't find the container with id 01d05415d0b8442725a7ec93e68ca9f91da4c5d9af1a572e9750a17e96f708ab
Jul 06 15:33:11.876581 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:11.876536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8" event={"ID":"8788900f-ff02-4349-96d0-b22184666734","Type":"ContainerStarted","Data":"01d05415d0b8442725a7ec93e68ca9f91da4c5d9af1a572e9750a17e96f708ab"}
Jul 06 15:33:12.256662 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.256574    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-864987f78f-gd98h"]
Jul 06 15:33:12.260516 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.260486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.263157 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.263120    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 15:33:12.263346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.263187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 15:33:12.263346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.263282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ck6s2\""
Jul 06 15:33:12.271052 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.271022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-864987f78f-gd98h"]
Jul 06 15:33:12.392150 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.392102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e32545-9622-448d-b622-15563ef383d4-tmp\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.392343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.392233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr89s\" (UniqueName: \"kubernetes.io/projected/b6e32545-9622-448d-b622-15563ef383d4-kube-api-access-rr89s\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.392343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.392279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e32545-9622-448d-b622-15563ef383d4-tls-certs\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.493769 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.493723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e32545-9622-448d-b622-15563ef383d4-tmp\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.493968 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.493838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr89s\" (UniqueName: \"kubernetes.io/projected/b6e32545-9622-448d-b622-15563ef383d4-kube-api-access-rr89s\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.493968 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.493883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e32545-9622-448d-b622-15563ef383d4-tls-certs\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.496309 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.496279    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6e32545-9622-448d-b622-15563ef383d4-tmp\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.496576 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.496553    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6e32545-9622-448d-b622-15563ef383d4-tls-certs\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.504240 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.504211    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr89s\" (UniqueName: \"kubernetes.io/projected/b6e32545-9622-448d-b622-15563ef383d4-kube-api-access-rr89s\") pod \"kube-auth-proxy-864987f78f-gd98h\" (UID: \"b6e32545-9622-448d-b622-15563ef383d4\") " pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:12.574127 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:12.574030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h"
Jul 06 15:33:13.190605 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.190572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-864987f78f-gd98h"]
Jul 06 15:33:13.193265 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:33:13.193222    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e32545_9622_448d_b622_15563ef383d4.slice/crio-172613a7da55c50bb3beac443f6440ae4d4761b9984227fdbc1c720741c133fd WatchSource:0}: Error finding container 172613a7da55c50bb3beac443f6440ae4d4761b9984227fdbc1c720741c133fd: Status 404 returned error can't find the container with id 172613a7da55c50bb3beac443f6440ae4d4761b9984227fdbc1c720741c133fd
Jul 06 15:33:13.425954 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.425904    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"]
Jul 06 15:33:13.429801 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.429775    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.431968 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.431945    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 15:33:13.432100 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.431945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 15:33:13.432100 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.431945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 15:33:13.432100 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.431945    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 15:33:13.432100 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.432077    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tscrk\""
Jul 06 15:33:13.446867 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.446834    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"]
Jul 06 15:33:13.504567 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-cacerts\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.504791 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcf5f\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-kube-api-access-pcf5f\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.504791 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-token\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.504791 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.504791 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.504791 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0163b-a35c-433e-8d5d-7b5c3392df53-local-certs\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.505001 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.504813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.605900 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.605863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-cacerts\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.605914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcf5f\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-kube-api-access-pcf5f\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.605971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-token\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.605990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.606015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.606058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0163b-a35c-433e-8d5d-7b5c3392df53-local-certs\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.606081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.606789 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.606762    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.608436 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.608391    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.608547 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.608433    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0163b-a35c-433e-8d5d-7b5c3392df53-local-certs\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.608641 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.608621    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-cacerts\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.608717 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.608662    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.617184 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.617144    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-token\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.617340 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.617302    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcf5f\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-kube-api-access-pcf5f\") pod \"istiod-openshift-gateway-7574459656-tsvxl\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.741837 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.741735    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:13.904307 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:13.904262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h" event={"ID":"b6e32545-9622-448d-b622-15563ef383d4","Type":"ContainerStarted","Data":"172613a7da55c50bb3beac443f6440ae4d4761b9984227fdbc1c720741c133fd"}
Jul 06 15:33:14.389374 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.389339    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"]
Jul 06 15:33:14.696117 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:33:14.696079    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b0163b_a35c_433e_8d5d_7b5c3392df53.slice/crio-858cfef94812cb0b4c3b0b8da26d1bf59182e82e824d381015ae93160f65f9c5 WatchSource:0}: Error finding container 858cfef94812cb0b4c3b0b8da26d1bf59182e82e824d381015ae93160f65f9c5: Status 404 returned error can't find the container with id 858cfef94812cb0b4c3b0b8da26d1bf59182e82e824d381015ae93160f65f9c5
Jul 06 15:33:14.912011 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.911973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8" event={"ID":"8788900f-ff02-4349-96d0-b22184666734","Type":"ContainerStarted","Data":"4ba5f47f9cb0801203a5aeb3ea0fa81d24d8efa3c10d57d8607f769112c9aff5"}
Jul 06 15:33:14.912225 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.912118    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:14.913892 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.913855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7" event={"ID":"1a3665d2-dfb6-4704-847a-4b9ee5a376a8","Type":"ContainerStarted","Data":"1319b5b5f7bd491952b0a6be384ae287f7a98794614069f6e0be5300f188814c"}
Jul 06 15:33:14.914094 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.913941    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:14.915235 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.915201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" event={"ID":"02b0163b-a35c-433e-8d5d-7b5c3392df53","Type":"ContainerStarted","Data":"858cfef94812cb0b4c3b0b8da26d1bf59182e82e824d381015ae93160f65f9c5"}
Jul 06 15:33:14.948318 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.948205    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8" podStartSLOduration=1.37056187 podStartE2EDuration="4.948186945s" totalImagesPullingTime="3.577625075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:14.945836694 +0000 UTC m=+386.758699473" watchObservedRunningTime="2026-07-06 15:33:14.948186945 +0000 UTC m=+386.761049723"
Jul 06 15:33:14.998206 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:14.998128    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7" podStartSLOduration=2.196260323 podStartE2EDuration="5.998099766s" totalImagesPullingTime="3.801839443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:14.995354076 +0000 UTC m=+386.808216873" watchObservedRunningTime="2026-07-06 15:33:14.998099766 +0000 UTC m=+386.810962546"
Jul 06 15:33:16.929374 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:16.929330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h" event={"ID":"b6e32545-9622-448d-b622-15563ef383d4","Type":"ContainerStarted","Data":"e6011ef12fdc7f3e8f6b0a104c6e90bdd7aa86ff235db38bc1857c251c89230d"}
Jul 06 15:33:16.947510 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:16.947430    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-864987f78f-gd98h" podStartSLOduration=1.637515652 podStartE2EDuration="4.9473877s" totalImagesPullingTime="3.309872048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:16.94689803 +0000 UTC m=+388.759760809" watchObservedRunningTime="2026-07-06 15:33:16.9473877 +0000 UTC m=+388.760250479"
Jul 06 15:33:19.174111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.174071    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 15:33:19.174445 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.174140    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 15:33:19.615741 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.615626    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"]
Jul 06 15:33:19.619285 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.619259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.623757 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.623725    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 15:33:19.623899 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.623738    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 15:33:19.624482 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.624455    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tgbnv\""
Jul 06 15:33:19.624482 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.624458    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 15:33:19.624854 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.624810    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 15:33:19.624854 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.624847    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 15:33:19.637287 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.637250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"]
Jul 06 15:33:19.773936 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.773879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-manager-config\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.773936 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.773939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-metrics-cert\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.774221 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.773977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cztqp\" (UniqueName: \"kubernetes.io/projected/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-kube-api-access-cztqp\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.774221 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.774091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-cert\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.875552 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.875515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-manager-config\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.875794 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.875566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-metrics-cert\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.875794 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.875604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cztqp\" (UniqueName: \"kubernetes.io/projected/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-kube-api-access-cztqp\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.875794 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.875680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-cert\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.876370 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.876347    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-manager-config\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.878350 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.878329    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-cert\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.878414 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.878379    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-metrics-cert\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.887386 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.887351    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cztqp\" (UniqueName: \"kubernetes.io/projected/0729f95a-995d-4c7f-99e1-fe51e18dcdbe-kube-api-access-cztqp\") pod \"lws-controller-manager-df4586b48-zngsx\" (UID: \"0729f95a-995d-4c7f-99e1-fe51e18dcdbe\") " pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.929922 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.929875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:19.950071 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.950031    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" event={"ID":"02b0163b-a35c-433e-8d5d-7b5c3392df53","Type":"ContainerStarted","Data":"31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041"}
Jul 06 15:33:19.950361 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:19.950339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:20.006608 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:20.005638    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" podStartSLOduration=2.5307526339999997 podStartE2EDuration="7.005614613s" totalImagesPullingTime="4.474861979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:20.004973655 +0000 UTC m=+391.817836433" watchObservedRunningTime="2026-07-06 15:33:20.005614613 +0000 UTC m=+391.818477394"
Jul 06 15:33:20.137613 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:20.137580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"]
Jul 06 15:33:20.139774 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:33:20.139733    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0729f95a_995d_4c7f_99e1_fe51e18dcdbe.slice/crio-5ab7dc92bc1d83edd352cf741fc4e0359c83bd93b5a995439718d0327e74d40d WatchSource:0}: Error finding container 5ab7dc92bc1d83edd352cf741fc4e0359c83bd93b5a995439718d0327e74d40d: Status 404 returned error can't find the container with id 5ab7dc92bc1d83edd352cf741fc4e0359c83bd93b5a995439718d0327e74d40d
Jul 06 15:33:20.958938 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:20.958877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx" event={"ID":"0729f95a-995d-4c7f-99e1-fe51e18dcdbe","Type":"ContainerStarted","Data":"5ab7dc92bc1d83edd352cf741fc4e0359c83bd93b5a995439718d0327e74d40d"}
Jul 06 15:33:20.961526 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:20.961493    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-tsvxl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 15:33:20.961683 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:20.961558    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" podUID="02b0163b-a35c-433e-8d5d-7b5c3392df53" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 15:33:21.965482 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:21.965376    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx" event={"ID":"0729f95a-995d-4c7f-99e1-fe51e18dcdbe","Type":"ContainerStarted","Data":"ae4f95b9205a8c65a988870595183d617b60f1b5f467b66db15c2f8ef8404049"}
Jul 06 15:33:21.965894 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:21.965493    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:33:21.991064 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:21.990996    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx" podStartSLOduration=1.44739666 podStartE2EDuration="2.990975232s" totalImagesPullingTime="1.543578572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:33:21.988378653 +0000 UTC m=+393.801241432" watchObservedRunningTime="2026-07-06 15:33:21.990975232 +0000 UTC m=+393.803838010"
Jul 06 15:33:22.903081 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:22.903038    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"]
Jul 06 15:33:22.903274 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:22.903234    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" podUID="02b0163b-a35c-433e-8d5d-7b5c3392df53" containerName="discovery" containerID="cri-o://31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041" gracePeriod=30
Jul 06 15:33:22.904042 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:22.903975    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:23.183962 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.183926    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:23.210049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210016    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0163b-a35c-433e-8d5d-7b5c3392df53-local-certs\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.210245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210098    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-cacerts\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.210245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210143    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcf5f\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-kube-api-access-pcf5f\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.210245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210172    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-dns-cert\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.210245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210231    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-kubeconfig\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.210476 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210281    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-ca-configmap\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.210476 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.210313    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-token\") pod \"02b0163b-a35c-433e-8d5d-7b5c3392df53\" (UID: \"02b0163b-a35c-433e-8d5d-7b5c3392df53\") "
Jul 06 15:33:23.213160 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.213120    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b0163b-a35c-433e-8d5d-7b5c3392df53-local-certs" (OuterVolumeSpecName: "local-certs") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:33:23.213312 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.213172    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:33:23.213312 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.213235    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-token" (OuterVolumeSpecName: "istio-token") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:33:23.213463 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.213441    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:33:23.214020 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.213984    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:33:23.215504 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.215470    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-cacerts" (OuterVolumeSpecName: "cacerts") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:33:23.215626 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.215572    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-kube-api-access-pcf5f" (OuterVolumeSpecName: "kube-api-access-pcf5f") pod "02b0163b-a35c-433e-8d5d-7b5c3392df53" (UID: "02b0163b-a35c-433e-8d5d-7b5c3392df53"). InnerVolumeSpecName "kube-api-access-pcf5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:33:23.311933 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311872    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-ca-configmap\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.311933 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311922    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-token\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.311933 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311933    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02b0163b-a35c-433e-8d5d-7b5c3392df53-local-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.311933 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311941    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-cacerts\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.311933 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311951    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcf5f\" (UniqueName: \"kubernetes.io/projected/02b0163b-a35c-433e-8d5d-7b5c3392df53-kube-api-access-pcf5f\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.312239 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311959    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-csr-dns-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.312239 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.311967    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02b0163b-a35c-433e-8d5d-7b5c3392df53-istio-kubeconfig\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:33:23.976816 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.976777    2580 generic.go:356] "Generic (PLEG): container finished" podID="02b0163b-a35c-433e-8d5d-7b5c3392df53" containerID="31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041" exitCode=0
Jul 06 15:33:23.977021 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.976837    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"
Jul 06 15:33:23.977021 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.976870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" event={"ID":"02b0163b-a35c-433e-8d5d-7b5c3392df53","Type":"ContainerDied","Data":"31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041"}
Jul 06 15:33:23.977021 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.976959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl" event={"ID":"02b0163b-a35c-433e-8d5d-7b5c3392df53","Type":"ContainerDied","Data":"858cfef94812cb0b4c3b0b8da26d1bf59182e82e824d381015ae93160f65f9c5"}
Jul 06 15:33:23.977021 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.976984    2580 scope.go:117] "RemoveContainer" containerID="31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041"
Jul 06 15:33:23.987474 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.987451    2580 scope.go:117] "RemoveContainer" containerID="31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041"
Jul 06 15:33:23.987839 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:33:23.987818    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041\": container with ID starting with 31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041 not found: ID does not exist" containerID="31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041"
Jul 06 15:33:23.987887 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:23.987852    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041"} err="failed to get container status \"31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041\": rpc error: code = NotFound desc = could not find container \"31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041\": container with ID starting with 31a069fe0fd38ce39c48c0ddfdd38f08bef616b48f64dec341dbe4c2f7667041 not found: ID does not exist"
Jul 06 15:33:24.002762 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:24.002723    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"]
Jul 06 15:33:24.008932 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:24.008894    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tsvxl"]
Jul 06 15:33:24.022612 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:24.022575    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02b0163b-a35c-433e-8d5d-7b5c3392df53" path="/var/lib/kubelet/pods/02b0163b-a35c-433e-8d5d-7b5c3392df53/volumes"
Jul 06 15:33:25.923522 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:25.923489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tqwl7"
Jul 06 15:33:25.925230 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:25.925209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8frj8"
Jul 06 15:33:32.973125 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:33:32.973090    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-df4586b48-zngsx"
Jul 06 15:34:20.931801 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:20.931754    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78bb99c546-zzlvc"]
Jul 06 15:34:26.296426 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.296390    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"]
Jul 06 15:34:26.296917 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.296825    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02b0163b-a35c-433e-8d5d-7b5c3392df53" containerName="discovery"
Jul 06 15:34:26.296917 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.296837    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="02b0163b-a35c-433e-8d5d-7b5c3392df53" containerName="discovery"
Jul 06 15:34:26.296917 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.296914    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="02b0163b-a35c-433e-8d5d-7b5c3392df53" containerName="discovery"
Jul 06 15:34:26.300318 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.300297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.302971 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.302948    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6q2bl\""
Jul 06 15:34:26.303182 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.303167    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 15:34:26.303640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.303621    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 15:34:26.320553 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.320506    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"]
Jul 06 15:34:26.379303 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.379262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjqn4\" (UniqueName: \"kubernetes.io/projected/e9e28b80-09ab-403c-8197-c6a95157993c-kube-api-access-qjqn4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.379503 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.379327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.379503 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.379395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.480114 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.480068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjqn4\" (UniqueName: \"kubernetes.io/projected/e9e28b80-09ab-403c-8197-c6a95157993c-kube-api-access-qjqn4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.480289 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.480154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.480289 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.480181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.480637 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.480617    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.480727 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.480634    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.489939 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.489910    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjqn4\" (UniqueName: \"kubernetes.io/projected/e9e28b80-09ab-403c-8197-c6a95157993c-kube-api-access-qjqn4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.610692 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.610598    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:26.760033 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:26.759995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"]
Jul 06 15:34:26.763161 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:34:26.763132    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e28b80_09ab_403c_8197_c6a95157993c.slice/crio-d7da8d4e3091ba135c140b27b645196ca725d6823261df426bc33cf3d19ac4ec WatchSource:0}: Error finding container d7da8d4e3091ba135c140b27b645196ca725d6823261df426bc33cf3d19ac4ec: Status 404 returned error can't find the container with id d7da8d4e3091ba135c140b27b645196ca725d6823261df426bc33cf3d19ac4ec
Jul 06 15:34:27.134086 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.134051    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"]
Jul 06 15:34:27.136894 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.136869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.148809 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.148777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"]
Jul 06 15:34:27.188233 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.188191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.188417 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.188243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m8lv\" (UniqueName: \"kubernetes.io/projected/08310f53-b9cd-4347-8270-c10f252ae27a-kube-api-access-5m8lv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.188417 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.188325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.288878 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.288836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m8lv\" (UniqueName: \"kubernetes.io/projected/08310f53-b9cd-4347-8270-c10f252ae27a-kube-api-access-5m8lv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.288878 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.288885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.289151 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.288957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.289346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.289310    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.289346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.289323    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.295262 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.295232    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e28b80-09ab-403c-8197-c6a95157993c" containerID="6859072ef96a8a4f95ee12d4ecfe04ad0965f64882b0e19868c2c1887e7de15c" exitCode=0
Jul 06 15:34:27.295410 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.295329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77" event={"ID":"e9e28b80-09ab-403c-8197-c6a95157993c","Type":"ContainerDied","Data":"6859072ef96a8a4f95ee12d4ecfe04ad0965f64882b0e19868c2c1887e7de15c"}
Jul 06 15:34:27.295410 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.295382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77" event={"ID":"e9e28b80-09ab-403c-8197-c6a95157993c","Type":"ContainerStarted","Data":"d7da8d4e3091ba135c140b27b645196ca725d6823261df426bc33cf3d19ac4ec"}
Jul 06 15:34:27.297644 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.297624    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m8lv\" (UniqueName: \"kubernetes.io/projected/08310f53-b9cd-4347-8270-c10f252ae27a-kube-api-access-5m8lv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.446685 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.446586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:27.496373 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.496337    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"]
Jul 06 15:34:27.499966 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.499923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.515016 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.514429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"]
Jul 06 15:34:27.591923 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.591889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.592105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.591984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmgdv\" (UniqueName: \"kubernetes.io/projected/52e0dde8-8d93-4803-8578-0dbf53035aef-kube-api-access-jmgdv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.592105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.592012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.599087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.599057    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"]
Jul 06 15:34:27.601136 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:34:27.601107    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08310f53_b9cd_4347_8270_c10f252ae27a.slice/crio-e075784da0e0d65c193654e5d46ef0955c8a335d09c1a3910ede57386ffb7716 WatchSource:0}: Error finding container e075784da0e0d65c193654e5d46ef0955c8a335d09c1a3910ede57386ffb7716: Status 404 returned error can't find the container with id e075784da0e0d65c193654e5d46ef0955c8a335d09c1a3910ede57386ffb7716
Jul 06 15:34:27.693162 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.693129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.693343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.693189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.693343 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.693277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmgdv\" (UniqueName: \"kubernetes.io/projected/52e0dde8-8d93-4803-8578-0dbf53035aef-kube-api-access-jmgdv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.693558 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.693541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.693640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.693614    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.703163 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.703135    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmgdv\" (UniqueName: \"kubernetes.io/projected/52e0dde8-8d93-4803-8578-0dbf53035aef-kube-api-access-jmgdv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.818294 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.818253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:27.914099 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.913999    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"]
Jul 06 15:34:27.918175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.918145    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:27.928111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.928081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"]
Jul 06 15:34:27.996143 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.996113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:27.996304 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.996170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf8k7\" (UniqueName: \"kubernetes.io/projected/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-kube-api-access-mf8k7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:27.996388 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:27.996308    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.016105 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.016026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"]
Jul 06 15:34:28.045754 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:34:28.045690    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e0dde8_8d93_4803_8578_0dbf53035aef.slice/crio-bc25dcf29d407a526cad0eb63f16b1332623ede256e84ff5d64416694e97ae8d WatchSource:0}: Error finding container bc25dcf29d407a526cad0eb63f16b1332623ede256e84ff5d64416694e97ae8d: Status 404 returned error can't find the container with id bc25dcf29d407a526cad0eb63f16b1332623ede256e84ff5d64416694e97ae8d
Jul 06 15:34:28.097685 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.097648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.097899 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.097751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.097899 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.097812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf8k7\" (UniqueName: \"kubernetes.io/projected/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-kube-api-access-mf8k7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.098148 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.098126    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.098193 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.098152    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.107276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.107246    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf8k7\" (UniqueName: \"kubernetes.io/projected/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-kube-api-access-mf8k7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.249448 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.249352    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:28.303617 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.303382    2580 generic.go:356] "Generic (PLEG): container finished" podID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerID="e25b8314e675e42f7d8797e3500b52f5322de15c98bab059fa20d75222f468a4" exitCode=0
Jul 06 15:34:28.303617 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.303549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw" event={"ID":"52e0dde8-8d93-4803-8578-0dbf53035aef","Type":"ContainerDied","Data":"e25b8314e675e42f7d8797e3500b52f5322de15c98bab059fa20d75222f468a4"}
Jul 06 15:34:28.303617 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.303604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw" event={"ID":"52e0dde8-8d93-4803-8578-0dbf53035aef","Type":"ContainerStarted","Data":"bc25dcf29d407a526cad0eb63f16b1332623ede256e84ff5d64416694e97ae8d"}
Jul 06 15:34:28.306974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.306755    2580 generic.go:356] "Generic (PLEG): container finished" podID="08310f53-b9cd-4347-8270-c10f252ae27a" containerID="d2a3c74fd3283f61d24ee21c112b4e4ecd3f0121408394d4d18c54327de0afd3" exitCode=0
Jul 06 15:34:28.306974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.306922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s" event={"ID":"08310f53-b9cd-4347-8270-c10f252ae27a","Type":"ContainerDied","Data":"d2a3c74fd3283f61d24ee21c112b4e4ecd3f0121408394d4d18c54327de0afd3"}
Jul 06 15:34:28.306974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.306983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s" event={"ID":"08310f53-b9cd-4347-8270-c10f252ae27a","Type":"ContainerStarted","Data":"e075784da0e0d65c193654e5d46ef0955c8a335d09c1a3910ede57386ffb7716"}
Jul 06 15:34:28.314250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.314139    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e28b80-09ab-403c-8197-c6a95157993c" containerID="6af469da3b614578be96539f435cd02face517b39e617c50936e3c1cbe93c000" exitCode=0
Jul 06 15:34:28.314250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.314180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77" event={"ID":"e9e28b80-09ab-403c-8197-c6a95157993c","Type":"ContainerDied","Data":"6af469da3b614578be96539f435cd02face517b39e617c50936e3c1cbe93c000"}
Jul 06 15:34:28.407665 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:28.407626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"]
Jul 06 15:34:28.409989 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:34:28.409958    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3df8aa2_76eb_4fdd_8d60_14f47bb68725.slice/crio-b82b60928105384fe0da4c8f676db5072ea1ed64c14eae9f257f48490485b503 WatchSource:0}: Error finding container b82b60928105384fe0da4c8f676db5072ea1ed64c14eae9f257f48490485b503: Status 404 returned error can't find the container with id b82b60928105384fe0da4c8f676db5072ea1ed64c14eae9f257f48490485b503
Jul 06 15:34:29.322350 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:29.322319    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerID="64051208ecc54c1a21aea3baddbd1f8528ebcae4dfd56188d62a5f2fa565ae26" exitCode=0
Jul 06 15:34:29.322819 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:29.322409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99" event={"ID":"a3df8aa2-76eb-4fdd-8d60-14f47bb68725","Type":"ContainerDied","Data":"64051208ecc54c1a21aea3baddbd1f8528ebcae4dfd56188d62a5f2fa565ae26"}
Jul 06 15:34:29.322819 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:29.322474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99" event={"ID":"a3df8aa2-76eb-4fdd-8d60-14f47bb68725","Type":"ContainerStarted","Data":"b82b60928105384fe0da4c8f676db5072ea1ed64c14eae9f257f48490485b503"}
Jul 06 15:34:29.325333 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:29.325295    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e28b80-09ab-403c-8197-c6a95157993c" containerID="dd45947a18c530285b739791f39dc1be7b5571c47f8fc4fc3844d85639f721ac" exitCode=0
Jul 06 15:34:29.325441 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:29.325338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77" event={"ID":"e9e28b80-09ab-403c-8197-c6a95157993c","Type":"ContainerDied","Data":"dd45947a18c530285b739791f39dc1be7b5571c47f8fc4fc3844d85639f721ac"}
Jul 06 15:34:30.332725 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.332670    2580 generic.go:356] "Generic (PLEG): container finished" podID="08310f53-b9cd-4347-8270-c10f252ae27a" containerID="2f2552c8029ec17e607ff86ca4e561a1a341951179165da763f3b0fb407238b0" exitCode=0
Jul 06 15:34:30.333179 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.332737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s" event={"ID":"08310f53-b9cd-4347-8270-c10f252ae27a","Type":"ContainerDied","Data":"2f2552c8029ec17e607ff86ca4e561a1a341951179165da763f3b0fb407238b0"}
Jul 06 15:34:30.334769 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.334745    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerID="273dc8a292c11543decbe8d81c1462749f0f18cbdac43ae16099fba82eac1eda" exitCode=0
Jul 06 15:34:30.334859 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.334831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99" event={"ID":"a3df8aa2-76eb-4fdd-8d60-14f47bb68725","Type":"ContainerDied","Data":"273dc8a292c11543decbe8d81c1462749f0f18cbdac43ae16099fba82eac1eda"}
Jul 06 15:34:30.336824 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.336802    2580 generic.go:356] "Generic (PLEG): container finished" podID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerID="e4c7ae307d65944ac5c633ff29b233ec927069b4abf0e9bd0c2544139eaa86d5" exitCode=0
Jul 06 15:34:30.336912 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.336870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw" event={"ID":"52e0dde8-8d93-4803-8578-0dbf53035aef","Type":"ContainerDied","Data":"e4c7ae307d65944ac5c633ff29b233ec927069b4abf0e9bd0c2544139eaa86d5"}
Jul 06 15:34:30.474108 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.474074    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:30.520089 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.520044    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-bundle\") pod \"e9e28b80-09ab-403c-8197-c6a95157993c\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") "
Jul 06 15:34:30.520261 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.520102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjqn4\" (UniqueName: \"kubernetes.io/projected/e9e28b80-09ab-403c-8197-c6a95157993c-kube-api-access-qjqn4\") pod \"e9e28b80-09ab-403c-8197-c6a95157993c\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") "
Jul 06 15:34:30.520261 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.520201    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-util\") pod \"e9e28b80-09ab-403c-8197-c6a95157993c\" (UID: \"e9e28b80-09ab-403c-8197-c6a95157993c\") "
Jul 06 15:34:30.520831 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.520799    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-bundle" (OuterVolumeSpecName: "bundle") pod "e9e28b80-09ab-403c-8197-c6a95157993c" (UID: "e9e28b80-09ab-403c-8197-c6a95157993c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:30.523342 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.523301    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e28b80-09ab-403c-8197-c6a95157993c-kube-api-access-qjqn4" (OuterVolumeSpecName: "kube-api-access-qjqn4") pod "e9e28b80-09ab-403c-8197-c6a95157993c" (UID: "e9e28b80-09ab-403c-8197-c6a95157993c"). InnerVolumeSpecName "kube-api-access-qjqn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:34:30.527963 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.527929    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-util" (OuterVolumeSpecName: "util") pod "e9e28b80-09ab-403c-8197-c6a95157993c" (UID: "e9e28b80-09ab-403c-8197-c6a95157993c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:30.621014 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.620975    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:30.621014 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.621008    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjqn4\" (UniqueName: \"kubernetes.io/projected/e9e28b80-09ab-403c-8197-c6a95157993c-kube-api-access-qjqn4\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:30.621014 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:30.621019    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9e28b80-09ab-403c-8197-c6a95157993c-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:31.344626 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.344582    2580 generic.go:356] "Generic (PLEG): container finished" podID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerID="8a93b4319591673cb5745f3ab131791b3f30376848304d397812063b461c3a23" exitCode=0
Jul 06 15:34:31.345062 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.344672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw" event={"ID":"52e0dde8-8d93-4803-8578-0dbf53035aef","Type":"ContainerDied","Data":"8a93b4319591673cb5745f3ab131791b3f30376848304d397812063b461c3a23"}
Jul 06 15:34:31.346901 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.346872    2580 generic.go:356] "Generic (PLEG): container finished" podID="08310f53-b9cd-4347-8270-c10f252ae27a" containerID="be69677027cc686ed841f13edac4ec94de5a5ea0d4d739d89cfed9aa985487ab" exitCode=0
Jul 06 15:34:31.347055 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.346947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s" event={"ID":"08310f53-b9cd-4347-8270-c10f252ae27a","Type":"ContainerDied","Data":"be69677027cc686ed841f13edac4ec94de5a5ea0d4d739d89cfed9aa985487ab"}
Jul 06 15:34:31.348950 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.348924    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77"
Jul 06 15:34:31.349087 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.348924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77" event={"ID":"e9e28b80-09ab-403c-8197-c6a95157993c","Type":"ContainerDied","Data":"d7da8d4e3091ba135c140b27b645196ca725d6823261df426bc33cf3d19ac4ec"}
Jul 06 15:34:31.349139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.349094    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7da8d4e3091ba135c140b27b645196ca725d6823261df426bc33cf3d19ac4ec"
Jul 06 15:34:31.351000 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.350974    2580 generic.go:356] "Generic (PLEG): container finished" podID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerID="821bb7f3493e30b64221fa07e31da730a4bc3ad21caebc65c167f4d80688f00c" exitCode=0
Jul 06 15:34:31.351117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:31.351022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99" event={"ID":"a3df8aa2-76eb-4fdd-8d60-14f47bb68725","Type":"ContainerDied","Data":"821bb7f3493e30b64221fa07e31da730a4bc3ad21caebc65c167f4d80688f00c"}
Jul 06 15:34:32.513000 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.512964    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:32.557843 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.557817    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:32.561181 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.561157    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:32.640419 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640387    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmgdv\" (UniqueName: \"kubernetes.io/projected/52e0dde8-8d93-4803-8578-0dbf53035aef-kube-api-access-jmgdv\") pod \"52e0dde8-8d93-4803-8578-0dbf53035aef\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") "
Jul 06 15:34:32.640569 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf8k7\" (UniqueName: \"kubernetes.io/projected/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-kube-api-access-mf8k7\") pod \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") "
Jul 06 15:34:32.640569 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640473    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-bundle\") pod \"08310f53-b9cd-4347-8270-c10f252ae27a\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") "
Jul 06 15:34:32.640675 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-util\") pod \"08310f53-b9cd-4347-8270-c10f252ae27a\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") "
Jul 06 15:34:32.640746 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-bundle\") pod \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") "
Jul 06 15:34:32.640809 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640758    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-util\") pod \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\" (UID: \"a3df8aa2-76eb-4fdd-8d60-14f47bb68725\") "
Jul 06 15:34:32.640863 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-bundle\") pod \"52e0dde8-8d93-4803-8578-0dbf53035aef\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") "
Jul 06 15:34:32.640863 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m8lv\" (UniqueName: \"kubernetes.io/projected/08310f53-b9cd-4347-8270-c10f252ae27a-kube-api-access-5m8lv\") pod \"08310f53-b9cd-4347-8270-c10f252ae27a\" (UID: \"08310f53-b9cd-4347-8270-c10f252ae27a\") "
Jul 06 15:34:32.640958 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.640892    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-util\") pod \"52e0dde8-8d93-4803-8578-0dbf53035aef\" (UID: \"52e0dde8-8d93-4803-8578-0dbf53035aef\") "
Jul 06 15:34:32.641263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.641094    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-bundle" (OuterVolumeSpecName: "bundle") pod "08310f53-b9cd-4347-8270-c10f252ae27a" (UID: "08310f53-b9cd-4347-8270-c10f252ae27a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:32.641263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.641193    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.641769 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.641670    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-bundle" (OuterVolumeSpecName: "bundle") pod "52e0dde8-8d93-4803-8578-0dbf53035aef" (UID: "52e0dde8-8d93-4803-8578-0dbf53035aef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:32.641769 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.641655    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-bundle" (OuterVolumeSpecName: "bundle") pod "a3df8aa2-76eb-4fdd-8d60-14f47bb68725" (UID: "a3df8aa2-76eb-4fdd-8d60-14f47bb68725"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:32.643249 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.643217    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-kube-api-access-mf8k7" (OuterVolumeSpecName: "kube-api-access-mf8k7") pod "a3df8aa2-76eb-4fdd-8d60-14f47bb68725" (UID: "a3df8aa2-76eb-4fdd-8d60-14f47bb68725"). InnerVolumeSpecName "kube-api-access-mf8k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:34:32.643365 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.643290    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08310f53-b9cd-4347-8270-c10f252ae27a-kube-api-access-5m8lv" (OuterVolumeSpecName: "kube-api-access-5m8lv") pod "08310f53-b9cd-4347-8270-c10f252ae27a" (UID: "08310f53-b9cd-4347-8270-c10f252ae27a"). InnerVolumeSpecName "kube-api-access-5m8lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:34:32.643365 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.643293    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e0dde8-8d93-4803-8578-0dbf53035aef-kube-api-access-jmgdv" (OuterVolumeSpecName: "kube-api-access-jmgdv") pod "52e0dde8-8d93-4803-8578-0dbf53035aef" (UID: "52e0dde8-8d93-4803-8578-0dbf53035aef"). InnerVolumeSpecName "kube-api-access-jmgdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:34:32.647620 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.647585    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-util" (OuterVolumeSpecName: "util") pod "52e0dde8-8d93-4803-8578-0dbf53035aef" (UID: "52e0dde8-8d93-4803-8578-0dbf53035aef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:32.649685 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.649653    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-util" (OuterVolumeSpecName: "util") pod "08310f53-b9cd-4347-8270-c10f252ae27a" (UID: "08310f53-b9cd-4347-8270-c10f252ae27a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:32.649953 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.649927    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-util" (OuterVolumeSpecName: "util") pod "a3df8aa2-76eb-4fdd-8d60-14f47bb68725" (UID: "a3df8aa2-76eb-4fdd-8d60-14f47bb68725"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:34:32.741960 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.741921    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08310f53-b9cd-4347-8270-c10f252ae27a-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.741960 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.741955    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.741960 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.741964    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.742192 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.741975    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.742192 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.741983    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m8lv\" (UniqueName: \"kubernetes.io/projected/08310f53-b9cd-4347-8270-c10f252ae27a-kube-api-access-5m8lv\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.742192 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.741992    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e0dde8-8d93-4803-8578-0dbf53035aef-util\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.742192 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.742001    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmgdv\" (UniqueName: \"kubernetes.io/projected/52e0dde8-8d93-4803-8578-0dbf53035aef-kube-api-access-jmgdv\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:32.742192 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:32.742010    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf8k7\" (UniqueName: \"kubernetes.io/projected/a3df8aa2-76eb-4fdd-8d60-14f47bb68725-kube-api-access-mf8k7\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:33.364063 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.364027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99"
Jul 06 15:34:33.364218 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.364026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99" event={"ID":"a3df8aa2-76eb-4fdd-8d60-14f47bb68725","Type":"ContainerDied","Data":"b82b60928105384fe0da4c8f676db5072ea1ed64c14eae9f257f48490485b503"}
Jul 06 15:34:33.364218 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.364184    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b82b60928105384fe0da4c8f676db5072ea1ed64c14eae9f257f48490485b503"
Jul 06 15:34:33.365919 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.365892    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw"
Jul 06 15:34:33.365919 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.365906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw" event={"ID":"52e0dde8-8d93-4803-8578-0dbf53035aef","Type":"ContainerDied","Data":"bc25dcf29d407a526cad0eb63f16b1332623ede256e84ff5d64416694e97ae8d"}
Jul 06 15:34:33.366111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.365954    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc25dcf29d407a526cad0eb63f16b1332623ede256e84ff5d64416694e97ae8d"
Jul 06 15:34:33.367711 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.367676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s" event={"ID":"08310f53-b9cd-4347-8270-c10f252ae27a","Type":"ContainerDied","Data":"e075784da0e0d65c193654e5d46ef0955c8a335d09c1a3910ede57386ffb7716"}
Jul 06 15:34:33.367819 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.367729    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s"
Jul 06 15:34:33.367819 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:33.367731    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e075784da0e0d65c193654e5d46ef0955c8a335d09c1a3910ede57386ffb7716"
Jul 06 15:34:45.954224 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:45.954179    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-78bb99c546-zzlvc" podUID="3b9f36ac-464d-4f32-bfec-6526fbe7640b" containerName="console" containerID="cri-o://9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d" gracePeriod=15
Jul 06 15:34:46.200462 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.200433    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:34:46.363520 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363426    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-oauth-config\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.363520 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363500    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-trusted-ca-bundle\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.363785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8dlv\" (UniqueName: \"kubernetes.io/projected/3b9f36ac-464d-4f32-bfec-6526fbe7640b-kube-api-access-s8dlv\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.363785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363542    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-oauth-serving-cert\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.363785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363567    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-service-ca\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.363785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-config\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.363785 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.363643    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-serving-cert\") pod \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\" (UID: \"3b9f36ac-464d-4f32-bfec-6526fbe7640b\") "
Jul 06 15:34:46.364057 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.364027    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-service-ca" (OuterVolumeSpecName: "service-ca") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:34:46.364117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.364045    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:34:46.364117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.364062    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-config" (OuterVolumeSpecName: "console-config") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:34:46.364117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.364052    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 15:34:46.365786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.365756    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:34:46.365882 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.365804    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:34:46.365882 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.365823    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9f36ac-464d-4f32-bfec-6526fbe7640b-kube-api-access-s8dlv" (OuterVolumeSpecName: "kube-api-access-s8dlv") pod "3b9f36ac-464d-4f32-bfec-6526fbe7640b" (UID: "3b9f36ac-464d-4f32-bfec-6526fbe7640b"). InnerVolumeSpecName "kube-api-access-s8dlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:34:46.435216 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.435180    2580 generic.go:356] "Generic (PLEG): container finished" podID="3b9f36ac-464d-4f32-bfec-6526fbe7640b" containerID="9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d" exitCode=0
Jul 06 15:34:46.435359 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.435227    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bb99c546-zzlvc"
Jul 06 15:34:46.435359 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.435263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bb99c546-zzlvc" event={"ID":"3b9f36ac-464d-4f32-bfec-6526fbe7640b","Type":"ContainerDied","Data":"9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d"}
Jul 06 15:34:46.435359 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.435313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bb99c546-zzlvc" event={"ID":"3b9f36ac-464d-4f32-bfec-6526fbe7640b","Type":"ContainerDied","Data":"cb93e55d664d326aafc21a7cf9d205f348bbe6fe333b7370f00b99bb10de20fb"}
Jul 06 15:34:46.435359 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.435333    2580 scope.go:117] "RemoveContainer" containerID="9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d"
Jul 06 15:34:46.445111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.445074    2580 scope.go:117] "RemoveContainer" containerID="9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d"
Jul 06 15:34:46.445465 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:34:46.445445    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d\": container with ID starting with 9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d not found: ID does not exist" containerID="9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d"
Jul 06 15:34:46.445526 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.445476    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d"} err="failed to get container status \"9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d\": rpc error: code = NotFound desc = could not find container \"9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d\": container with ID starting with 9e803e1f249c7a6f72a8991bf843fb228614040b72ef8720f305481401a5fa1d not found: ID does not exist"
Jul 06 15:34:46.459452 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.459413    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78bb99c546-zzlvc"]
Jul 06 15:34:46.463641 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.463606    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78bb99c546-zzlvc"]
Jul 06 15:34:46.465339 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465306    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-config\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:46.465339 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465331    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-serving-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:46.465339 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465341    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b9f36ac-464d-4f32-bfec-6526fbe7640b-console-oauth-config\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:46.465533 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465351    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-trusted-ca-bundle\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:46.465533 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465360    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8dlv\" (UniqueName: \"kubernetes.io/projected/3b9f36ac-464d-4f32-bfec-6526fbe7640b-kube-api-access-s8dlv\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:46.465533 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465369    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-oauth-serving-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:46.465533 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:46.465377    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b9f36ac-464d-4f32-bfec-6526fbe7640b-service-ca\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:34:48.022913 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:48.022871    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b9f36ac-464d-4f32-bfec-6526fbe7640b" path="/var/lib/kubelet/pods/3b9f36ac-464d-4f32-bfec-6526fbe7640b/volumes"
Jul 06 15:34:58.866802 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.866763    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"]
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867158    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867170    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867181    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867186    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867195    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9f36ac-464d-4f32-bfec-6526fbe7640b" containerName="console"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867201    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9f36ac-464d-4f32-bfec-6526fbe7640b" containerName="console"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867207    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="pull"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867213    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="pull"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867228    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867235    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867246    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="extract"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867250    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="extract"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867258    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867263    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="util"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867271    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="pull"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867276    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="pull"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867284    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="extract"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="extract"
Jul 06 15:34:58.867290 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867295    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="pull"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867303    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="pull"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867314    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867319    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867327    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867331    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867340    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="pull"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867346    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="pull"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867413    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e0dde8-8d93-4803-8578-0dbf53035aef" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867423    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9f36ac-464d-4f32-bfec-6526fbe7640b" containerName="console"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867432    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e28b80-09ab-403c-8197-c6a95157993c" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867443    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="08310f53-b9cd-4347-8270-c10f252ae27a" containerName="extract"
Jul 06 15:34:58.867872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.867452    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3df8aa2-76eb-4fdd-8d60-14f47bb68725" containerName="extract"
Jul 06 15:34:58.872188 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.872162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.877583 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.877552    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 15:34:58.879614 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.879589    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 06 15:34:58.879766 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.879679    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 06 15:34:58.880466 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.880449    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 15:34:58.882795 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.882774    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6q2bl\""
Jul 06 15:34:58.882901 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.882802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2049b8-4c34-4c1e-a206-205fa2188401-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.882901 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.882828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t69rm\" (UniqueName: \"kubernetes.io/projected/3e2049b8-4c34-4c1e-a206-205fa2188401-kube-api-access-t69rm\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.883005 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.882914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e2049b8-4c34-4c1e-a206-205fa2188401-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.909983 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.909944    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"]
Jul 06 15:34:58.983253 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.983218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t69rm\" (UniqueName: \"kubernetes.io/projected/3e2049b8-4c34-4c1e-a206-205fa2188401-kube-api-access-t69rm\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.983456 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.983310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e2049b8-4c34-4c1e-a206-205fa2188401-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.983456 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.983336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2049b8-4c34-4c1e-a206-205fa2188401-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.984073 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.984048    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3e2049b8-4c34-4c1e-a206-205fa2188401-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:58.985879 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:58.985847    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2049b8-4c34-4c1e-a206-205fa2188401-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:59.007276 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:59.007243    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t69rm\" (UniqueName: \"kubernetes.io/projected/3e2049b8-4c34-4c1e-a206-205fa2188401-kube-api-access-t69rm\") pod \"kuadrant-console-plugin-66b5774985-qlb8k\" (UID: \"3e2049b8-4c34-4c1e-a206-205fa2188401\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:59.182354 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:59.182315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"
Jul 06 15:34:59.354799 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:59.354767    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k"]
Jul 06 15:34:59.358161 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:34:59.358123    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e2049b8_4c34_4c1e_a206_205fa2188401.slice/crio-d794a585398a434d93e519b65a058eb8f68031c738eee9d0eac00c902c500277 WatchSource:0}: Error finding container d794a585398a434d93e519b65a058eb8f68031c738eee9d0eac00c902c500277: Status 404 returned error can't find the container with id d794a585398a434d93e519b65a058eb8f68031c738eee9d0eac00c902c500277
Jul 06 15:34:59.506819 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:34:59.506728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k" event={"ID":"3e2049b8-4c34-4c1e-a206-205fa2188401","Type":"ContainerStarted","Data":"d794a585398a434d93e519b65a058eb8f68031c738eee9d0eac00c902c500277"}
Jul 06 15:35:24.660332 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:24.660296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k" event={"ID":"3e2049b8-4c34-4c1e-a206-205fa2188401","Type":"ContainerStarted","Data":"92df225b154a6a474b737d659c590e5f22fa97bc63a41aeec6bb13f4f2bb9b46"}
Jul 06 15:35:24.686596 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:24.686536    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qlb8k" podStartSLOduration=2.380922715 podStartE2EDuration="26.686502214s" totalImagesPullingTime="24.305579499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:24.68336607 +0000 UTC m=+516.496228849" watchObservedRunningTime="2026-07-06 15:35:24.686502214 +0000 UTC m=+516.499364991"
Jul 06 15:35:43.469928 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.469840    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-q7b8x"]
Jul 06 15:35:43.472417 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.472397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:43.476964 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.476936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l4n2b\""
Jul 06 15:35:43.486021 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.485992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7b8x"]
Jul 06 15:35:43.595115 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.595083    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6qls\" (UniqueName: \"kubernetes.io/projected/e8d16fb2-8873-4fd7-a6a0-320ae083a379-kube-api-access-m6qls\") pod \"authorino-58cd955675-q7b8x\" (UID: \"e8d16fb2-8873-4fd7-a6a0-320ae083a379\") " pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:43.696081 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.696037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6qls\" (UniqueName: \"kubernetes.io/projected/e8d16fb2-8873-4fd7-a6a0-320ae083a379-kube-api-access-m6qls\") pod \"authorino-58cd955675-q7b8x\" (UID: \"e8d16fb2-8873-4fd7-a6a0-320ae083a379\") " pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:43.707670 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.707634    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6qls\" (UniqueName: \"kubernetes.io/projected/e8d16fb2-8873-4fd7-a6a0-320ae083a379-kube-api-access-m6qls\") pod \"authorino-58cd955675-q7b8x\" (UID: \"e8d16fb2-8873-4fd7-a6a0-320ae083a379\") " pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:43.782407 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.782313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:43.836584 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.836553    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-nldxk"]
Jul 06 15:35:43.839315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.839294    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:35:43.858212 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.858083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nldxk"]
Jul 06 15:35:43.937226 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.937202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7b8x"]
Jul 06 15:35:43.939639 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:35:43.939606    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8d16fb2_8873_4fd7_a6a0_320ae083a379.slice/crio-ca8fca5439e0a0b7213154a9e754b8dd615dc63db52a8d5a607d65fa34e5da3c WatchSource:0}: Error finding container ca8fca5439e0a0b7213154a9e754b8dd615dc63db52a8d5a607d65fa34e5da3c: Status 404 returned error can't find the container with id ca8fca5439e0a0b7213154a9e754b8dd615dc63db52a8d5a607d65fa34e5da3c
Jul 06 15:35:43.998448 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:43.998406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cnrv\" (UniqueName: \"kubernetes.io/projected/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe-kube-api-access-7cnrv\") pod \"authorino-75d556bd77-nldxk\" (UID: \"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe\") " pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:35:44.099947 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:44.099851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cnrv\" (UniqueName: \"kubernetes.io/projected/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe-kube-api-access-7cnrv\") pod \"authorino-75d556bd77-nldxk\" (UID: \"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe\") " pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:35:44.110196 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:44.110164    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cnrv\" (UniqueName: \"kubernetes.io/projected/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe-kube-api-access-7cnrv\") pod \"authorino-75d556bd77-nldxk\" (UID: \"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe\") " pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:35:44.151822 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:44.151785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:35:44.290152 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:44.290126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nldxk"]
Jul 06 15:35:44.292092 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:35:44.292064    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ac66f6b_17cb_4cd2_a950_a98fdf51b1fe.slice/crio-02ef5679895db68119e901442293336b12a224d07b14cdda3b565bdf91fd78a6 WatchSource:0}: Error finding container 02ef5679895db68119e901442293336b12a224d07b14cdda3b565bdf91fd78a6: Status 404 returned error can't find the container with id 02ef5679895db68119e901442293336b12a224d07b14cdda3b565bdf91fd78a6
Jul 06 15:35:44.771571 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:44.771524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nldxk" event={"ID":"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe","Type":"ContainerStarted","Data":"02ef5679895db68119e901442293336b12a224d07b14cdda3b565bdf91fd78a6"}
Jul 06 15:35:44.772904 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:44.772867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7b8x" event={"ID":"e8d16fb2-8873-4fd7-a6a0-320ae083a379","Type":"ContainerStarted","Data":"ca8fca5439e0a0b7213154a9e754b8dd615dc63db52a8d5a607d65fa34e5da3c"}
Jul 06 15:35:47.802386 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:47.802340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nldxk" event={"ID":"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe","Type":"ContainerStarted","Data":"1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068"}
Jul 06 15:35:47.804400 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:47.804370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7b8x" event={"ID":"e8d16fb2-8873-4fd7-a6a0-320ae083a379","Type":"ContainerStarted","Data":"0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00"}
Jul 06 15:35:47.829071 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:47.829014    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-nldxk" podStartSLOduration=1.479621185 podStartE2EDuration="4.828999009s" totalImagesPullingTime="3.349377824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:47.827526176 +0000 UTC m=+539.640388973" watchObservedRunningTime="2026-07-06 15:35:47.828999009 +0000 UTC m=+539.641861785"
Jul 06 15:35:47.849588 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:47.849529    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-q7b8x" podStartSLOduration=1.158735137 podStartE2EDuration="4.849513149s" totalImagesPullingTime="3.690778012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:35:47.848255683 +0000 UTC m=+539.661118461" watchObservedRunningTime="2026-07-06 15:35:47.849513149 +0000 UTC m=+539.662375925"
Jul 06 15:35:47.881935 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:47.881844    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7b8x"]
Jul 06 15:35:49.816079 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:49.816036    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-q7b8x" podUID="e8d16fb2-8873-4fd7-a6a0-320ae083a379" containerName="authorino" containerID="cri-o://0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00" gracePeriod=30
Jul 06 15:35:50.069098 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.069027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:50.157269 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.157212    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6qls\" (UniqueName: \"kubernetes.io/projected/e8d16fb2-8873-4fd7-a6a0-320ae083a379-kube-api-access-m6qls\") pod \"e8d16fb2-8873-4fd7-a6a0-320ae083a379\" (UID: \"e8d16fb2-8873-4fd7-a6a0-320ae083a379\") "
Jul 06 15:35:50.159413 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.159369    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d16fb2-8873-4fd7-a6a0-320ae083a379-kube-api-access-m6qls" (OuterVolumeSpecName: "kube-api-access-m6qls") pod "e8d16fb2-8873-4fd7-a6a0-320ae083a379" (UID: "e8d16fb2-8873-4fd7-a6a0-320ae083a379"). InnerVolumeSpecName "kube-api-access-m6qls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:35:50.258352 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.258302    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6qls\" (UniqueName: \"kubernetes.io/projected/e8d16fb2-8873-4fd7-a6a0-320ae083a379-kube-api-access-m6qls\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:35:50.823144 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.823103    2580 generic.go:356] "Generic (PLEG): container finished" podID="e8d16fb2-8873-4fd7-a6a0-320ae083a379" containerID="0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00" exitCode=0
Jul 06 15:35:50.823563 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.823150    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q7b8x"
Jul 06 15:35:50.823563 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.823184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7b8x" event={"ID":"e8d16fb2-8873-4fd7-a6a0-320ae083a379","Type":"ContainerDied","Data":"0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00"}
Jul 06 15:35:50.823563 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.823231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q7b8x" event={"ID":"e8d16fb2-8873-4fd7-a6a0-320ae083a379","Type":"ContainerDied","Data":"ca8fca5439e0a0b7213154a9e754b8dd615dc63db52a8d5a607d65fa34e5da3c"}
Jul 06 15:35:50.823563 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.823249    2580 scope.go:117] "RemoveContainer" containerID="0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00"
Jul 06 15:35:50.832550 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.832528    2580 scope.go:117] "RemoveContainer" containerID="0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00"
Jul 06 15:35:50.832868 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:35:50.832850    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00\": container with ID starting with 0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00 not found: ID does not exist" containerID="0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00"
Jul 06 15:35:50.832932 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.832877    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00"} err="failed to get container status \"0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00\": rpc error: code = NotFound desc = could not find container \"0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00\": container with ID starting with 0f603af52be6009d86e5cfed26cbf70ef5b52b25bbdff8f5fcf1393c998a7f00 not found: ID does not exist"
Jul 06 15:35:50.851283 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.851240    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7b8x"]
Jul 06 15:35:50.860913 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:50.860878    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-q7b8x"]
Jul 06 15:35:52.022402 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:35:52.022357    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8d16fb2-8873-4fd7-a6a0-320ae083a379" path="/var/lib/kubelet/pods/e8d16fb2-8873-4fd7-a6a0-320ae083a379/volumes"
Jul 06 15:36:16.989882 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:16.989841    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l5h5v"]
Jul 06 15:36:16.990303 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:16.990202    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8d16fb2-8873-4fd7-a6a0-320ae083a379" containerName="authorino"
Jul 06 15:36:16.990303 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:16.990214    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8d16fb2-8873-4fd7-a6a0-320ae083a379" containerName="authorino"
Jul 06 15:36:16.990303 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:16.990289    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8d16fb2-8873-4fd7-a6a0-320ae083a379" containerName="authorino"
Jul 06 15:36:16.993509 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:16.993487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:16.999300 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:16.999271    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l5h5v"]
Jul 06 15:36:17.104403 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.104356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76mwb\" (UniqueName: \"kubernetes.io/projected/3d5f042d-0994-42da-84ba-317e5d2d2d70-kube-api-access-76mwb\") pod \"authorino-5d4b67c446-l5h5v\" (UID: \"3d5f042d-0994-42da-84ba-317e5d2d2d70\") " pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:17.205806 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.205765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76mwb\" (UniqueName: \"kubernetes.io/projected/3d5f042d-0994-42da-84ba-317e5d2d2d70-kube-api-access-76mwb\") pod \"authorino-5d4b67c446-l5h5v\" (UID: \"3d5f042d-0994-42da-84ba-317e5d2d2d70\") " pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:17.218357 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.218315    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76mwb\" (UniqueName: \"kubernetes.io/projected/3d5f042d-0994-42da-84ba-317e5d2d2d70-kube-api-access-76mwb\") pod \"authorino-5d4b67c446-l5h5v\" (UID: \"3d5f042d-0994-42da-84ba-317e5d2d2d70\") " pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:17.225402 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.225357    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l5h5v"]
Jul 06 15:36:17.225690 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.225666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:17.257556 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.257470    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-864f899cc9-877x7"]
Jul 06 15:36:17.266742 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.266499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.269407 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.269378    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 15:36:17.270533 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.270503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-864f899cc9-877x7"]
Jul 06 15:36:17.373497 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.373468    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l5h5v"]
Jul 06 15:36:17.375804 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:36:17.375772    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d5f042d_0994_42da_84ba_317e5d2d2d70.slice/crio-976fdcdadc9b1a780b95e65e51c9f3dc6ef0caf76717b3bd8657b3d0957149fd WatchSource:0}: Error finding container 976fdcdadc9b1a780b95e65e51c9f3dc6ef0caf76717b3bd8657b3d0957149fd: Status 404 returned error can't find the container with id 976fdcdadc9b1a780b95e65e51c9f3dc6ef0caf76717b3bd8657b3d0957149fd
Jul 06 15:36:17.408139 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.408095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18f1def5-625b-47f8-8a6e-82c8eb516820-tls-cert\") pod \"authorino-864f899cc9-877x7\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") " pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.408316 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.408168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q28vd\" (UniqueName: \"kubernetes.io/projected/18f1def5-625b-47f8-8a6e-82c8eb516820-kube-api-access-q28vd\") pod \"authorino-864f899cc9-877x7\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") " pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.509170 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.509077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q28vd\" (UniqueName: \"kubernetes.io/projected/18f1def5-625b-47f8-8a6e-82c8eb516820-kube-api-access-q28vd\") pod \"authorino-864f899cc9-877x7\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") " pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.509170 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.509145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18f1def5-625b-47f8-8a6e-82c8eb516820-tls-cert\") pod \"authorino-864f899cc9-877x7\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") " pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.511675 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.511647    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18f1def5-625b-47f8-8a6e-82c8eb516820-tls-cert\") pod \"authorino-864f899cc9-877x7\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") " pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.518829 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.518782    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q28vd\" (UniqueName: \"kubernetes.io/projected/18f1def5-625b-47f8-8a6e-82c8eb516820-kube-api-access-q28vd\") pod \"authorino-864f899cc9-877x7\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") " pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.581046 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.581006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:36:17.724040 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.724017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-864f899cc9-877x7"]
Jul 06 15:36:17.726237 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:36:17.726209    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18f1def5_625b_47f8_8a6e_82c8eb516820.slice/crio-c1a8ddf2b4c178406034d4c06dba80f97038959a88c9998abada8ae2b8eb88ac WatchSource:0}: Error finding container c1a8ddf2b4c178406034d4c06dba80f97038959a88c9998abada8ae2b8eb88ac: Status 404 returned error can't find the container with id c1a8ddf2b4c178406034d4c06dba80f97038959a88c9998abada8ae2b8eb88ac
Jul 06 15:36:17.966044 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.966007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864f899cc9-877x7" event={"ID":"18f1def5-625b-47f8-8a6e-82c8eb516820","Type":"ContainerStarted","Data":"c1a8ddf2b4c178406034d4c06dba80f97038959a88c9998abada8ae2b8eb88ac"}
Jul 06 15:36:17.967757 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.967688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l5h5v" event={"ID":"3d5f042d-0994-42da-84ba-317e5d2d2d70","Type":"ContainerStarted","Data":"43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4"}
Jul 06 15:36:17.967939 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.967767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l5h5v" event={"ID":"3d5f042d-0994-42da-84ba-317e5d2d2d70","Type":"ContainerStarted","Data":"976fdcdadc9b1a780b95e65e51c9f3dc6ef0caf76717b3bd8657b3d0957149fd"}
Jul 06 15:36:17.967939 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.967809    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-l5h5v" podUID="3d5f042d-0994-42da-84ba-317e5d2d2d70" containerName="authorino" containerID="cri-o://43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4" gracePeriod=30
Jul 06 15:36:17.986532 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:17.986440    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-l5h5v" podStartSLOduration=1.495419741 podStartE2EDuration="1.986423479s" totalImagesPullingTime="491.003738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:36:17.983950327 +0000 UTC m=+569.796813103" watchObservedRunningTime="2026-07-06 15:36:17.986423479 +0000 UTC m=+569.799286256"
Jul 06 15:36:18.218585 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.218550    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:18.316233 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.316197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76mwb\" (UniqueName: \"kubernetes.io/projected/3d5f042d-0994-42da-84ba-317e5d2d2d70-kube-api-access-76mwb\") pod \"3d5f042d-0994-42da-84ba-317e5d2d2d70\" (UID: \"3d5f042d-0994-42da-84ba-317e5d2d2d70\") "
Jul 06 15:36:18.318356 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.318315    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5f042d-0994-42da-84ba-317e5d2d2d70-kube-api-access-76mwb" (OuterVolumeSpecName: "kube-api-access-76mwb") pod "3d5f042d-0994-42da-84ba-317e5d2d2d70" (UID: "3d5f042d-0994-42da-84ba-317e5d2d2d70"). InnerVolumeSpecName "kube-api-access-76mwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:36:18.417824 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.417734    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76mwb\" (UniqueName: \"kubernetes.io/projected/3d5f042d-0994-42da-84ba-317e5d2d2d70-kube-api-access-76mwb\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:36:18.974670 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.974631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864f899cc9-877x7" event={"ID":"18f1def5-625b-47f8-8a6e-82c8eb516820","Type":"ContainerStarted","Data":"ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2"}
Jul 06 15:36:18.976003 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.975976    2580 generic.go:356] "Generic (PLEG): container finished" podID="3d5f042d-0994-42da-84ba-317e5d2d2d70" containerID="43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4" exitCode=0
Jul 06 15:36:18.976135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.976013    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-l5h5v"
Jul 06 15:36:18.976135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.976056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l5h5v" event={"ID":"3d5f042d-0994-42da-84ba-317e5d2d2d70","Type":"ContainerDied","Data":"43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4"}
Jul 06 15:36:18.976135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.976102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-l5h5v" event={"ID":"3d5f042d-0994-42da-84ba-317e5d2d2d70","Type":"ContainerDied","Data":"976fdcdadc9b1a780b95e65e51c9f3dc6ef0caf76717b3bd8657b3d0957149fd"}
Jul 06 15:36:18.976135 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.976123    2580 scope.go:117] "RemoveContainer" containerID="43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4"
Jul 06 15:36:18.985897 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.985875    2580 scope.go:117] "RemoveContainer" containerID="43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4"
Jul 06 15:36:18.986235 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:36:18.986217    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4\": container with ID starting with 43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4 not found: ID does not exist" containerID="43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4"
Jul 06 15:36:18.986293 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.986250    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4"} err="failed to get container status \"43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4\": rpc error: code = NotFound desc = could not find container \"43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4\": container with ID starting with 43f55b5d26092a700939a1d6d0ee1006604dbae96d188ca88176dc2454ed4af4 not found: ID does not exist"
Jul 06 15:36:18.993635 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:18.993579    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-864f899cc9-877x7" podStartSLOduration=1.618120313 podStartE2EDuration="1.993560286s" totalImagesPullingTime="375.439973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:36:18.992243552 +0000 UTC m=+570.805106330" watchObservedRunningTime="2026-07-06 15:36:18.993560286 +0000 UTC m=+570.806423063"
Jul 06 15:36:19.008346 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.008308    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l5h5v"]
Jul 06 15:36:19.015126 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.015090    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-l5h5v"]
Jul 06 15:36:19.023750 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.023691    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nldxk"]
Jul 06 15:36:19.023961 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.023929    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-nldxk" podUID="8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" containerName="authorino" containerID="cri-o://1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068" gracePeriod=30
Jul 06 15:36:19.289426 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.289394    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:36:19.426761 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.426714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cnrv\" (UniqueName: \"kubernetes.io/projected/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe-kube-api-access-7cnrv\") pod \"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe\" (UID: \"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe\") "
Jul 06 15:36:19.428824 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.428786    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe-kube-api-access-7cnrv" (OuterVolumeSpecName: "kube-api-access-7cnrv") pod "8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" (UID: "8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe"). InnerVolumeSpecName "kube-api-access-7cnrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:36:19.528010 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.527917    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cnrv\" (UniqueName: \"kubernetes.io/projected/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe-kube-api-access-7cnrv\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:36:19.984051 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.984017    2580 generic.go:356] "Generic (PLEG): container finished" podID="8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" containerID="1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068" exitCode=0
Jul 06 15:36:19.984246 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.984062    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-nldxk"
Jul 06 15:36:19.984246 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.984102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nldxk" event={"ID":"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe","Type":"ContainerDied","Data":"1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068"}
Jul 06 15:36:19.984246 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.984146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-nldxk" event={"ID":"8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe","Type":"ContainerDied","Data":"02ef5679895db68119e901442293336b12a224d07b14cdda3b565bdf91fd78a6"}
Jul 06 15:36:19.984246 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.984166    2580 scope.go:117] "RemoveContainer" containerID="1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068"
Jul 06 15:36:19.993312 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.993293    2580 scope.go:117] "RemoveContainer" containerID="1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068"
Jul 06 15:36:19.993608 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:36:19.993589    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068\": container with ID starting with 1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068 not found: ID does not exist" containerID="1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068"
Jul 06 15:36:19.993670 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:19.993622    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068"} err="failed to get container status \"1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068\": rpc error: code = NotFound desc = could not find container \"1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068\": container with ID starting with 1c146d433e895377bb8a9194f89f2d76aa59bd9e51b3111c943b82cfb3c7f068 not found: ID does not exist"
Jul 06 15:36:20.006267 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:20.006236    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nldxk"]
Jul 06 15:36:20.008646 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:20.008618    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-nldxk"]
Jul 06 15:36:20.027772 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:20.027735    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5f042d-0994-42da-84ba-317e5d2d2d70" path="/var/lib/kubelet/pods/3d5f042d-0994-42da-84ba-317e5d2d2d70/volumes"
Jul 06 15:36:20.028090 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:20.028077    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" path="/var/lib/kubelet/pods/8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe/volumes"
Jul 06 15:36:23.777426 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777390    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79684c65f9-4fxpm"]
Jul 06 15:36:23.777898 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777826    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5f042d-0994-42da-84ba-317e5d2d2d70" containerName="authorino"
Jul 06 15:36:23.777898 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777841    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5f042d-0994-42da-84ba-317e5d2d2d70" containerName="authorino"
Jul 06 15:36:23.777898 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777867    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" containerName="authorino"
Jul 06 15:36:23.777898 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777876    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" containerName="authorino"
Jul 06 15:36:23.778046 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777951    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5f042d-0994-42da-84ba-317e5d2d2d70" containerName="authorino"
Jul 06 15:36:23.778046 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.777959    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ac66f6b-17cb-4cd2-a950-a98fdf51b1fe" containerName="authorino"
Jul 06 15:36:23.782591 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.782568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:23.784920 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.784892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 15:36:23.785063 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.785010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 15:36:23.785117 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.785060    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tbd4d\""
Jul 06 15:36:23.792293 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.792266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79684c65f9-4fxpm"]
Jul 06 15:36:23.865860 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.865827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42dkg\" (UniqueName: \"kubernetes.io/projected/35067147-c5e7-460d-9d5f-050fb4bb3d59-kube-api-access-42dkg\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:23.866055 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.865883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:23.966990 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.966946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42dkg\" (UniqueName: \"kubernetes.io/projected/35067147-c5e7-460d-9d5f-050fb4bb3d59-kube-api-access-42dkg\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:23.967206 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.967031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:23.967206 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:36:23.967192    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 06 15:36:23.967324 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:36:23.967259    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert podName:35067147-c5e7-460d-9d5f-050fb4bb3d59 nodeName:}" failed. No retries permitted until 2026-07-06 15:36:24.467237731 +0000 UTC m=+576.280100507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert") pod "maas-controller-79684c65f9-4fxpm" (UID: "35067147-c5e7-460d-9d5f-050fb4bb3d59") : secret "maas-controller-webhook-cert" not found
Jul 06 15:36:23.975536 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:23.975488    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42dkg\" (UniqueName: \"kubernetes.io/projected/35067147-c5e7-460d-9d5f-050fb4bb3d59-kube-api-access-42dkg\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:24.472420 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:24.472383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:24.474864 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:24.474835    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert\") pod \"maas-controller-79684c65f9-4fxpm\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") " pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:24.694633 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:24.694593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:24.843530 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:24.843494    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79684c65f9-4fxpm"]
Jul 06 15:36:24.847043 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:36:24.847011    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35067147_c5e7_460d_9d5f_050fb4bb3d59.slice/crio-81f1590e4a43261b1696726c4e362185c45d775c2e6d92919d7c9854552f0fff WatchSource:0}: Error finding container 81f1590e4a43261b1696726c4e362185c45d775c2e6d92919d7c9854552f0fff: Status 404 returned error can't find the container with id 81f1590e4a43261b1696726c4e362185c45d775c2e6d92919d7c9854552f0fff
Jul 06 15:36:25.015040 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:25.014947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79684c65f9-4fxpm" event={"ID":"35067147-c5e7-460d-9d5f-050fb4bb3d59","Type":"ContainerStarted","Data":"81f1590e4a43261b1696726c4e362185c45d775c2e6d92919d7c9854552f0fff"}
Jul 06 15:36:28.035739 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:28.035670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79684c65f9-4fxpm" event={"ID":"35067147-c5e7-460d-9d5f-050fb4bb3d59","Type":"ContainerStarted","Data":"3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1"}
Jul 06 15:36:28.036114 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:28.035760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:28.064366 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:28.064307    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79684c65f9-4fxpm" podStartSLOduration=2.5988878079999997 podStartE2EDuration="5.064291308s" totalImagesPullingTime="2.4654035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:36:28.060677088 +0000 UTC m=+579.873539865" watchObservedRunningTime="2026-07-06 15:36:28.064291308 +0000 UTC m=+579.877154084"
Jul 06 15:36:38.579965 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:38.575974    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79684c65f9-4fxpm"]
Jul 06 15:36:38.579965 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:38.576313    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-79684c65f9-4fxpm" podUID="35067147-c5e7-460d-9d5f-050fb4bb3d59" containerName="manager" containerID="cri-o://3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1" gracePeriod=30
Jul 06 15:36:38.582813 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:38.582788    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:38.829217 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:38.829189    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:39.011049 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.011011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert\") pod \"35067147-c5e7-460d-9d5f-050fb4bb3d59\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") "
Jul 06 15:36:39.011279 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.011070    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42dkg\" (UniqueName: \"kubernetes.io/projected/35067147-c5e7-460d-9d5f-050fb4bb3d59-kube-api-access-42dkg\") pod \"35067147-c5e7-460d-9d5f-050fb4bb3d59\" (UID: \"35067147-c5e7-460d-9d5f-050fb4bb3d59\") "
Jul 06 15:36:39.013408 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.013375    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "35067147-c5e7-460d-9d5f-050fb4bb3d59" (UID: "35067147-c5e7-460d-9d5f-050fb4bb3d59"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:36:39.013508 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.013483    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35067147-c5e7-460d-9d5f-050fb4bb3d59-kube-api-access-42dkg" (OuterVolumeSpecName: "kube-api-access-42dkg") pod "35067147-c5e7-460d-9d5f-050fb4bb3d59" (UID: "35067147-c5e7-460d-9d5f-050fb4bb3d59"). InnerVolumeSpecName "kube-api-access-42dkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:36:39.098762 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.098688    2580 generic.go:356] "Generic (PLEG): container finished" podID="35067147-c5e7-460d-9d5f-050fb4bb3d59" containerID="3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1" exitCode=0
Jul 06 15:36:39.098762 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.098761    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79684c65f9-4fxpm"
Jul 06 15:36:39.098974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.098732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79684c65f9-4fxpm" event={"ID":"35067147-c5e7-460d-9d5f-050fb4bb3d59","Type":"ContainerDied","Data":"3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1"}
Jul 06 15:36:39.098974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.098846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79684c65f9-4fxpm" event={"ID":"35067147-c5e7-460d-9d5f-050fb4bb3d59","Type":"ContainerDied","Data":"81f1590e4a43261b1696726c4e362185c45d775c2e6d92919d7c9854552f0fff"}
Jul 06 15:36:39.098974 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.098878    2580 scope.go:117] "RemoveContainer" containerID="3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1"
Jul 06 15:36:39.108629 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.108605    2580 scope.go:117] "RemoveContainer" containerID="3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1"
Jul 06 15:36:39.108989 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:36:39.108968    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1\": container with ID starting with 3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1 not found: ID does not exist" containerID="3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1"
Jul 06 15:36:39.109065 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.109003    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1"} err="failed to get container status \"3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1\": rpc error: code = NotFound desc = could not find container \"3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1\": container with ID starting with 3be35c819e4564011a5e7be58bf1a0d25513ec048614344168a493c1665443e1 not found: ID does not exist"
Jul 06 15:36:39.112429 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.112405    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/35067147-c5e7-460d-9d5f-050fb4bb3d59-webhook-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:36:39.112487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.112432    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42dkg\" (UniqueName: \"kubernetes.io/projected/35067147-c5e7-460d-9d5f-050fb4bb3d59-kube-api-access-42dkg\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:36:39.125876 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.125836    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79684c65f9-4fxpm"]
Jul 06 15:36:39.130295 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:39.130256    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79684c65f9-4fxpm"]
Jul 06 15:36:40.022947 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:40.022905    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35067147-c5e7-460d-9d5f-050fb4bb3d59" path="/var/lib/kubelet/pods/35067147-c5e7-460d-9d5f-050fb4bb3d59/volumes"
Jul 06 15:36:50.108091 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:50.098247    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:36:50.108091 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:36:50.098542    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:37:00.459259 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.459210    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-t6jb5"]
Jul 06 15:37:00.459763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.459589    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35067147-c5e7-460d-9d5f-050fb4bb3d59" containerName="manager"
Jul 06 15:37:00.459763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.459600    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="35067147-c5e7-460d-9d5f-050fb4bb3d59" containerName="manager"
Jul 06 15:37:00.459763 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.459675    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="35067147-c5e7-460d-9d5f-050fb4bb3d59" containerName="manager"
Jul 06 15:37:00.466245 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.466215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.468998 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.468953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q4hmh\""
Jul 06 15:37:00.469186 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.469153    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 15:37:00.470773 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.470743    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-t6jb5"]
Jul 06 15:37:00.610828 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.610770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qft8l\" (UniqueName: \"kubernetes.io/projected/bfa257ca-1566-4621-97ea-53e85de26e19-kube-api-access-qft8l\") pod \"payload-pre-processing-7964457696-t6jb5\" (UID: \"bfa257ca-1566-4621-97ea-53e85de26e19\") " pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.611068 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.610881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa257ca-1566-4621-97ea-53e85de26e19-plugins-config-volume\") pod \"payload-pre-processing-7964457696-t6jb5\" (UID: \"bfa257ca-1566-4621-97ea-53e85de26e19\") " pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.712263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.712167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qft8l\" (UniqueName: \"kubernetes.io/projected/bfa257ca-1566-4621-97ea-53e85de26e19-kube-api-access-qft8l\") pod \"payload-pre-processing-7964457696-t6jb5\" (UID: \"bfa257ca-1566-4621-97ea-53e85de26e19\") " pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.712263 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.712211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa257ca-1566-4621-97ea-53e85de26e19-plugins-config-volume\") pod \"payload-pre-processing-7964457696-t6jb5\" (UID: \"bfa257ca-1566-4621-97ea-53e85de26e19\") " pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.712940 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.712922    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfa257ca-1566-4621-97ea-53e85de26e19-plugins-config-volume\") pod \"payload-pre-processing-7964457696-t6jb5\" (UID: \"bfa257ca-1566-4621-97ea-53e85de26e19\") " pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.720323 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.720289    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qft8l\" (UniqueName: \"kubernetes.io/projected/bfa257ca-1566-4621-97ea-53e85de26e19-kube-api-access-qft8l\") pod \"payload-pre-processing-7964457696-t6jb5\" (UID: \"bfa257ca-1566-4621-97ea-53e85de26e19\") " pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.778396 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.778351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:00.928394 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.928369    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-t6jb5"]
Jul 06 15:37:00.930568 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:37:00.930540    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa257ca_1566_4621_97ea_53e85de26e19.slice/crio-b0ad91f6f289f2b0a3f81d91d18eadc259b62bfd2cd8eb59d7fe34fde284699f WatchSource:0}: Error finding container b0ad91f6f289f2b0a3f81d91d18eadc259b62bfd2cd8eb59d7fe34fde284699f: Status 404 returned error can't find the container with id b0ad91f6f289f2b0a3f81d91d18eadc259b62bfd2cd8eb59d7fe34fde284699f
Jul 06 15:37:00.932492 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:00.932472    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 15:37:01.226539 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:01.226503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5" event={"ID":"bfa257ca-1566-4621-97ea-53e85de26e19","Type":"ContainerStarted","Data":"b0ad91f6f289f2b0a3f81d91d18eadc259b62bfd2cd8eb59d7fe34fde284699f"}
Jul 06 15:37:03.239480 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:03.239441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5" event={"ID":"bfa257ca-1566-4621-97ea-53e85de26e19","Type":"ContainerStarted","Data":"b8d2f64e7a9580d36d98ee54b68559317bf5b20f68c8cc0075f8d773ba3b478d"}
Jul 06 15:37:03.239973 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:03.239519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:03.258827 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:03.258773    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5" podStartSLOduration=1.545321006 podStartE2EDuration="3.258756059s" totalImagesPullingTime="1.713435053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:37:03.256886245 +0000 UTC m=+615.069749034" watchObservedRunningTime="2026-07-06 15:37:03.258756059 +0000 UTC m=+615.071618838"
Jul 06 15:37:14.248088 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:14.248003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-t6jb5"
Jul 06 15:37:26.291250 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.291208    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"]
Jul 06 15:37:26.294984 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.294957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.297461 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.297439    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 15:37:26.298419 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.298397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 15:37:26.298502 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.298417    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 15:37:26.298502 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.298406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f47hh\""
Jul 06 15:37:26.326938 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.326903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"]
Jul 06 15:37:26.341594 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.341555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.341594 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.341600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.341872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.341629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlpgp\" (UniqueName: \"kubernetes.io/projected/5b08b09c-9997-42e4-a431-ad4e838e9cee-kube-api-access-qlpgp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.341872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.341663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b08b09c-9997-42e4-a431-ad4e838e9cee-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.341872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.341717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.341872 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.341785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlpgp\" (UniqueName: \"kubernetes.io/projected/5b08b09c-9997-42e4-a431-ad4e838e9cee-kube-api-access-qlpgp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443315 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b08b09c-9997-42e4-a431-ad4e838e9cee-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443659 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443659 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443659 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443659 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443874 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443919 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443901    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.443981 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.443920    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.445782 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.445760    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b08b09c-9997-42e4-a431-ad4e838e9cee-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.446182 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.446164    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b08b09c-9997-42e4-a431-ad4e838e9cee-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.457119 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.457086    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlpgp\" (UniqueName: \"kubernetes.io/projected/5b08b09c-9997-42e4-a431-ad4e838e9cee-kube-api-access-qlpgp\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl\" (UID: \"5b08b09c-9997-42e4-a431-ad4e838e9cee\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.605640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.605545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:26.760079 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:26.760050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"]
Jul 06 15:37:26.762372 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:37:26.762337    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b08b09c_9997_42e4_a431_ad4e838e9cee.slice/crio-599dfaa4f6f249ee450b9b2c89d6821113e06eb4a08e801c6715584e346fcc5d WatchSource:0}: Error finding container 599dfaa4f6f249ee450b9b2c89d6821113e06eb4a08e801c6715584e346fcc5d: Status 404 returned error can't find the container with id 599dfaa4f6f249ee450b9b2c89d6821113e06eb4a08e801c6715584e346fcc5d
Jul 06 15:37:27.374524 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:27.374443    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl" event={"ID":"5b08b09c-9997-42e4-a431-ad4e838e9cee","Type":"ContainerStarted","Data":"599dfaa4f6f249ee450b9b2c89d6821113e06eb4a08e801c6715584e346fcc5d"}
Jul 06 15:37:28.382118 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:28.382083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl" event={"ID":"5b08b09c-9997-42e4-a431-ad4e838e9cee","Type":"ContainerStarted","Data":"e59fe38b62de9079573e2c04fb1bf35a318d8dc1df839a3b151a6356cfb24daf"}
Jul 06 15:37:28.406879 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:28.406821    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl" podStartSLOduration=1.134660851 podStartE2EDuration="2.406802812s" totalImagesPullingTime="1.272141961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:37:28.405085764 +0000 UTC m=+640.217948543" watchObservedRunningTime="2026-07-06 15:37:28.406802812 +0000 UTC m=+640.219665616"
Jul 06 15:37:36.606610 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:36.606570    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:36.607041 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:36.606643    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:36.619222 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:36.619192    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:37.446352 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:37.446317    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl"
Jul 06 15:37:44.800362 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.800325    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"]
Jul 06 15:37:44.803919 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.803898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:44.812111 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.812087    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 06 15:37:44.820623 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.820584    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"]
Jul 06 15:37:44.911131 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.911090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:44.911131 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.911127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:44.911363 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.911180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:44.911363 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.911216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:44.911363 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.911289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5hqp\" (UniqueName: \"kubernetes.io/projected/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-kube-api-access-s5hqp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:44.911363 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:44.911320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012288 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012288 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012612 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012612 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012612 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5hqp\" (UniqueName: \"kubernetes.io/projected/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-kube-api-access-s5hqp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012612 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012832 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012749    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012928 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012908    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.012971 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.012939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.014686 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.014665    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.014908 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.014889    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.022029 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.021997    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5hqp\" (UniqueName: \"kubernetes.io/projected/902b2e7c-5c0b-4d1a-a6fd-566c44004a98-kube-api-access-s5hqp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvht5\" (UID: \"902b2e7c-5c0b-4d1a-a6fd-566c44004a98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.114286 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.114204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:45.264752 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.264725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"]
Jul 06 15:37:45.266898 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:37:45.266867    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902b2e7c_5c0b_4d1a_a6fd_566c44004a98.slice/crio-e5dffcb4f94e9640d7c1ddab84c733f77d8e0fcd4961de40e7c0cfdadd17c8d5 WatchSource:0}: Error finding container e5dffcb4f94e9640d7c1ddab84c733f77d8e0fcd4961de40e7c0cfdadd17c8d5: Status 404 returned error can't find the container with id e5dffcb4f94e9640d7c1ddab84c733f77d8e0fcd4961de40e7c0cfdadd17c8d5
Jul 06 15:37:45.478900 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:45.478864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5" event={"ID":"902b2e7c-5c0b-4d1a-a6fd-566c44004a98","Type":"ContainerStarted","Data":"e5dffcb4f94e9640d7c1ddab84c733f77d8e0fcd4961de40e7c0cfdadd17c8d5"}
Jul 06 15:37:46.490381 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:46.490347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5" event={"ID":"902b2e7c-5c0b-4d1a-a6fd-566c44004a98","Type":"ContainerStarted","Data":"f1f6ae5b1bf16cb62471fe42e9e6543cb7087d2ac926e802b43f6f9287a5f99c"}
Jul 06 15:37:46.510227 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:46.510163    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5" podStartSLOduration=2.246110382 podStartE2EDuration="2.510142603s" totalImagesPullingTime="264.032221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:37:46.508670407 +0000 UTC m=+658.321533186" watchObservedRunningTime="2026-07-06 15:37:46.510142603 +0000 UTC m=+658.323005407"
Jul 06 15:37:55.114426 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:55.114383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:55.114931 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:55.114452    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:55.127295 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:55.127264    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:37:55.553959 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:37:55.553924    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvht5"
Jul 06 15:38:12.175150 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.175112    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"]
Jul 06 15:38:12.181713 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.181675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.184384 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.184354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 06 15:38:12.198856 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.198819    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"]
Jul 06 15:38:12.253368 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.253334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.253368 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.253372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.253614 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.253402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.253614 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.253471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6jjr\" (UniqueName: \"kubernetes.io/projected/d9bac989-594a-4b62-b074-e474309f294e-kube-api-access-c6jjr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.253614 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.253520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.253614 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.253550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9bac989-594a-4b62-b074-e474309f294e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354101 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354101 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354337 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354337 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6jjr\" (UniqueName: \"kubernetes.io/projected/d9bac989-594a-4b62-b074-e474309f294e-kube-api-access-c6jjr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354337 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354337 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9bac989-594a-4b62-b074-e474309f294e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354552 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354527    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354611 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354585    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.354756 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.354643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.356561 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.356526    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9bac989-594a-4b62-b074-e474309f294e-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.356674 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.356631    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9bac989-594a-4b62-b074-e474309f294e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.364313 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.364290    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6jjr\" (UniqueName: \"kubernetes.io/projected/d9bac989-594a-4b62-b074-e474309f294e-kube-api-access-c6jjr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n\" (UID: \"d9bac989-594a-4b62-b074-e474309f294e\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.498196 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.498098    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:12.644798 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:12.644771    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"]
Jul 06 15:38:12.646098 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:38:12.646071    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9bac989_594a_4b62_b074_e474309f294e.slice/crio-e693f37c747f27573382a0e09d820c3e37a2f5a5d7fac9979416381e1dfc6a18 WatchSource:0}: Error finding container e693f37c747f27573382a0e09d820c3e37a2f5a5d7fac9979416381e1dfc6a18: Status 404 returned error can't find the container with id e693f37c747f27573382a0e09d820c3e37a2f5a5d7fac9979416381e1dfc6a18
Jul 06 15:38:13.642145 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:13.642104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n" event={"ID":"d9bac989-594a-4b62-b074-e474309f294e","Type":"ContainerStarted","Data":"fb33252ab4d3adae3733f992814ed6aae8eaafe8dc767f80e03ef918350472b2"}
Jul 06 15:38:13.642557 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:13.642155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n" event={"ID":"d9bac989-594a-4b62-b074-e474309f294e","Type":"ContainerStarted","Data":"e693f37c747f27573382a0e09d820c3e37a2f5a5d7fac9979416381e1dfc6a18"}
Jul 06 15:38:13.660878 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:13.660819    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n" podStartSLOduration=1.448766652 podStartE2EDuration="1.660802258s" totalImagesPullingTime="212.035606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:38:13.659524212 +0000 UTC m=+685.472386990" watchObservedRunningTime="2026-07-06 15:38:13.660802258 +0000 UTC m=+685.473665059"
Jul 06 15:38:22.499005 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:22.498965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:22.499494 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:22.499236    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:22.511842 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:22.511814    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:38:22.702123 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:38:22.702087    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n"
Jul 06 15:39:02.803067 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.802978    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54c8db6ff8-q2cw6"]
Jul 06 15:39:02.806786 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.806763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:02.818141 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.818111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8d7bf7b4-3b68-44f3-bb19-9940880d1966-tls-cert\") pod \"authorino-54c8db6ff8-q2cw6\" (UID: \"8d7bf7b4-3b68-44f3-bb19-9940880d1966\") " pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:02.818324 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.818144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfcv6\" (UniqueName: \"kubernetes.io/projected/8d7bf7b4-3b68-44f3-bb19-9940880d1966-kube-api-access-hfcv6\") pod \"authorino-54c8db6ff8-q2cw6\" (UID: \"8d7bf7b4-3b68-44f3-bb19-9940880d1966\") " pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:02.818842 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.818818    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54c8db6ff8-q2cw6"]
Jul 06 15:39:02.919396 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.919359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfcv6\" (UniqueName: \"kubernetes.io/projected/8d7bf7b4-3b68-44f3-bb19-9940880d1966-kube-api-access-hfcv6\") pod \"authorino-54c8db6ff8-q2cw6\" (UID: \"8d7bf7b4-3b68-44f3-bb19-9940880d1966\") " pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:02.919583 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.919481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8d7bf7b4-3b68-44f3-bb19-9940880d1966-tls-cert\") pod \"authorino-54c8db6ff8-q2cw6\" (UID: \"8d7bf7b4-3b68-44f3-bb19-9940880d1966\") " pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:02.922147 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.922111    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8d7bf7b4-3b68-44f3-bb19-9940880d1966-tls-cert\") pod \"authorino-54c8db6ff8-q2cw6\" (UID: \"8d7bf7b4-3b68-44f3-bb19-9940880d1966\") " pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:02.928513 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:02.928475    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfcv6\" (UniqueName: \"kubernetes.io/projected/8d7bf7b4-3b68-44f3-bb19-9940880d1966-kube-api-access-hfcv6\") pod \"authorino-54c8db6ff8-q2cw6\" (UID: \"8d7bf7b4-3b68-44f3-bb19-9940880d1966\") " pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:03.117159 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.117071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54c8db6ff8-q2cw6"
Jul 06 15:39:03.267512 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.267478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54c8db6ff8-q2cw6"]
Jul 06 15:39:03.270183 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:39:03.270142    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d7bf7b4_3b68_44f3_bb19_9940880d1966.slice/crio-81519bc83e83dae6de7ee9a553f9ee75a24523fc0ba2a1103667d426dd176f78 WatchSource:0}: Error finding container 81519bc83e83dae6de7ee9a553f9ee75a24523fc0ba2a1103667d426dd176f78: Status 404 returned error can't find the container with id 81519bc83e83dae6de7ee9a553f9ee75a24523fc0ba2a1103667d426dd176f78
Jul 06 15:39:03.933201 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.933159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54c8db6ff8-q2cw6" event={"ID":"8d7bf7b4-3b68-44f3-bb19-9940880d1966","Type":"ContainerStarted","Data":"e9ed147c30d507c587eefed8675e336d7a9484ecd086d66ba1ed048da5c0b255"}
Jul 06 15:39:03.933628 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.933216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54c8db6ff8-q2cw6" event={"ID":"8d7bf7b4-3b68-44f3-bb19-9940880d1966","Type":"ContainerStarted","Data":"81519bc83e83dae6de7ee9a553f9ee75a24523fc0ba2a1103667d426dd176f78"}
Jul 06 15:39:03.954400 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.954332    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54c8db6ff8-q2cw6" podStartSLOduration=1.632823924 podStartE2EDuration="1.954308531s" totalImagesPullingTime="321.484607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:39:03.951849982 +0000 UTC m=+735.764712785" watchObservedRunningTime="2026-07-06 15:39:03.954308531 +0000 UTC m=+735.767171311"
Jul 06 15:39:03.994588 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.994535    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-864f899cc9-877x7"]
Jul 06 15:39:03.995037 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:03.995006    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-864f899cc9-877x7" podUID="18f1def5-625b-47f8-8a6e-82c8eb516820" containerName="authorino" containerID="cri-o://ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2" gracePeriod=30
Jul 06 15:39:04.270765 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.270623    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:39:04.332893 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.332859    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18f1def5-625b-47f8-8a6e-82c8eb516820-tls-cert\") pod \"18f1def5-625b-47f8-8a6e-82c8eb516820\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") "
Jul 06 15:39:04.332893 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.332897    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q28vd\" (UniqueName: \"kubernetes.io/projected/18f1def5-625b-47f8-8a6e-82c8eb516820-kube-api-access-q28vd\") pod \"18f1def5-625b-47f8-8a6e-82c8eb516820\" (UID: \"18f1def5-625b-47f8-8a6e-82c8eb516820\") "
Jul 06 15:39:04.335152 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.335112    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f1def5-625b-47f8-8a6e-82c8eb516820-kube-api-access-q28vd" (OuterVolumeSpecName: "kube-api-access-q28vd") pod "18f1def5-625b-47f8-8a6e-82c8eb516820" (UID: "18f1def5-625b-47f8-8a6e-82c8eb516820"). InnerVolumeSpecName "kube-api-access-q28vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:39:04.344495 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.344452    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18f1def5-625b-47f8-8a6e-82c8eb516820-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "18f1def5-625b-47f8-8a6e-82c8eb516820" (UID: "18f1def5-625b-47f8-8a6e-82c8eb516820"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:39:04.434432 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.434397    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/18f1def5-625b-47f8-8a6e-82c8eb516820-tls-cert\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:39:04.434432 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.434427    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q28vd\" (UniqueName: \"kubernetes.io/projected/18f1def5-625b-47f8-8a6e-82c8eb516820-kube-api-access-q28vd\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:39:04.941001 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.940966    2580 generic.go:356] "Generic (PLEG): container finished" podID="18f1def5-625b-47f8-8a6e-82c8eb516820" containerID="ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2" exitCode=0
Jul 06 15:39:04.941487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.941007    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-864f899cc9-877x7"
Jul 06 15:39:04.941487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.941053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864f899cc9-877x7" event={"ID":"18f1def5-625b-47f8-8a6e-82c8eb516820","Type":"ContainerDied","Data":"ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2"}
Jul 06 15:39:04.941487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.941106    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-864f899cc9-877x7" event={"ID":"18f1def5-625b-47f8-8a6e-82c8eb516820","Type":"ContainerDied","Data":"c1a8ddf2b4c178406034d4c06dba80f97038959a88c9998abada8ae2b8eb88ac"}
Jul 06 15:39:04.941487 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.941130    2580 scope.go:117] "RemoveContainer" containerID="ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2"
Jul 06 15:39:04.951831 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.951802    2580 scope.go:117] "RemoveContainer" containerID="ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2"
Jul 06 15:39:04.952158 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:39:04.952138    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2\": container with ID starting with ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2 not found: ID does not exist" containerID="ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2"
Jul 06 15:39:04.952214 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.952170    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2"} err="failed to get container status \"ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2\": rpc error: code = NotFound desc = could not find container \"ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2\": container with ID starting with ef9dfe2e093dbc9c1d8d66c6c44fc7c73e4ba689764bdbabeb35d338ef528dc2 not found: ID does not exist"
Jul 06 15:39:04.999860 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:04.999829    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-864f899cc9-877x7"]
Jul 06 15:39:05.005529 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:05.005497    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-864f899cc9-877x7"]
Jul 06 15:39:06.023210 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:39:06.023173    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f1def5-625b-47f8-8a6e-82c8eb516820" path="/var/lib/kubelet/pods/18f1def5-625b-47f8-8a6e-82c8eb516820/volumes"
Jul 06 15:41:50.139996 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:41:50.139963    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:41:50.142405 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:41:50.139973    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:46:50.176215 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:46:50.176097    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:46:50.186124 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:46:50.184354    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:51:50.220665 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:51:50.220515    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:51:50.224962 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:51:50.224059    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:56:50.257595 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:56:50.257468    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:56:50.264838 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:56:50.263412    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 15:58:36.563817 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.563773    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"]
Jul 06 15:58:36.564450 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.564385    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f1def5-625b-47f8-8a6e-82c8eb516820" containerName="authorino"
Jul 06 15:58:36.564450 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.564408    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f1def5-625b-47f8-8a6e-82c8eb516820" containerName="authorino"
Jul 06 15:58:36.564573 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.564515    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f1def5-625b-47f8-8a6e-82c8eb516820" containerName="authorino"
Jul 06 15:58:36.567750 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.567724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.570863 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.570828    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-37068508\"/\"openshift-service-ca.crt\""
Jul 06 15:58:36.570863 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.570840    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-37068508\"/\"kube-root-ca.crt\""
Jul 06 15:58:36.571077 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.570873    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-37068508\"/\"default-dockercfg-pb8f7\""
Jul 06 15:58:36.571077 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.570873    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-37068508\"/\"e2e-lifecycle-model-37068508-kserve-self-signed-certs\""
Jul 06 15:58:36.578396 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.578366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"]
Jul 06 15:58:36.712340 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.712295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-dshm\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.712528 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.712348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tmp-dir\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.712528 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.712437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-home\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.712528 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.712507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-model-cache\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.712640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.712544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrmlj\" (UniqueName: \"kubernetes.io/projected/4a2f47d0-a94f-4eac-80cd-1327f82b827f-kube-api-access-qrmlj\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.712640 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.712569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tls-certs\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.813896 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.813796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tmp-dir\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.813896 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.813849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-home\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.813896 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.813896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-model-cache\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.814175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.813925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrmlj\" (UniqueName: \"kubernetes.io/projected/4a2f47d0-a94f-4eac-80cd-1327f82b827f-kube-api-access-qrmlj\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.814175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.813942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tls-certs\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.814175 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.813994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-dshm\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.814325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.814273    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tmp-dir\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.814325 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.814291    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-model-cache\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.814410 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.814328    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-home\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.816378 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.816354    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-dshm\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.816570 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.816550    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tls-certs\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.822483 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.822452    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrmlj\" (UniqueName: \"kubernetes.io/projected/4a2f47d0-a94f-4eac-80cd-1327f82b827f-kube-api-access-qrmlj\") pod \"e2e-lifecycle-model-37068508-kserve-6678646775-98rqj\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") " pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:36.878557 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:36.878506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:37.026919 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:37.026879    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"]
Jul 06 15:58:37.032098 ip-10-0-141-57 kubenswrapper[2580]: W0706 15:58:37.032058    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a2f47d0_a94f_4eac_80cd_1327f82b827f.slice/crio-eed356bf62d8b18c44e7ee69d5a3e8c487f11b6f09446d7d71db41dfbe4de9c2 WatchSource:0}: Error finding container eed356bf62d8b18c44e7ee69d5a3e8c487f11b6f09446d7d71db41dfbe4de9c2: Status 404 returned error can't find the container with id eed356bf62d8b18c44e7ee69d5a3e8c487f11b6f09446d7d71db41dfbe4de9c2
Jul 06 15:58:37.661593 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:37.661558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj" event={"ID":"4a2f47d0-a94f-4eac-80cd-1327f82b827f","Type":"ContainerStarted","Data":"48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95"}
Jul 06 15:58:37.662020 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:37.661601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj" event={"ID":"4a2f47d0-a94f-4eac-80cd-1327f82b827f","Type":"ContainerStarted","Data":"eed356bf62d8b18c44e7ee69d5a3e8c487f11b6f09446d7d71db41dfbe4de9c2"}
Jul 06 15:58:37.681363 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:37.681301    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj" podStartSLOduration=1.68128163 podStartE2EDuration="1.68128163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 15:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 15:58:37.679445409 +0000 UTC m=+1909.492308210" watchObservedRunningTime="2026-07-06 15:58:37.68128163 +0000 UTC m=+1909.494144409"
Jul 06 15:58:46.879138 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:46.879094    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:46.879560 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:46.879153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:46.892063 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:46.892033    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:58:47.727596 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:58:47.727553    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:59:08.548900 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:08.548852    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"]
Jul 06 15:59:23.570103 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.570037    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj" podUID="4a2f47d0-a94f-4eac-80cd-1327f82b827f" containerName="main" containerID="cri-o://48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95" gracePeriod=45
Jul 06 15:59:23.826794 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.826717    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:59:23.927360 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.927328    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a2f47d0-a94f-4eac-80cd-1327f82b827f" containerID="48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95" exitCode=0
Jul 06 15:59:23.927556 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.927366    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"
Jul 06 15:59:23.927556 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.927408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj" event={"ID":"4a2f47d0-a94f-4eac-80cd-1327f82b827f","Type":"ContainerDied","Data":"48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95"}
Jul 06 15:59:23.927556 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.927461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj" event={"ID":"4a2f47d0-a94f-4eac-80cd-1327f82b827f","Type":"ContainerDied","Data":"eed356bf62d8b18c44e7ee69d5a3e8c487f11b6f09446d7d71db41dfbe4de9c2"}
Jul 06 15:59:23.927556 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.927484    2580 scope.go:117] "RemoveContainer" containerID="48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95"
Jul 06 15:59:23.937211 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.937171    2580 scope.go:117] "RemoveContainer" containerID="48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95"
Jul 06 15:59:23.937553 ip-10-0-141-57 kubenswrapper[2580]: E0706 15:59:23.937533    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95\": container with ID starting with 48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95 not found: ID does not exist" containerID="48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95"
Jul 06 15:59:23.937629 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.937564    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95"} err="failed to get container status \"48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95\": rpc error: code = NotFound desc = could not find container \"48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95\": container with ID starting with 48d790a4fd88c9b8d7129d1248d01bbbf68e64ced71db8ce8e35bb178d5c7b95 not found: ID does not exist"
Jul 06 15:59:23.956951 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.956916    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tls-certs\") pod \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") "
Jul 06 15:59:23.957125 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.956976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-dshm\") pod \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") "
Jul 06 15:59:23.957125 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.956995    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tmp-dir\") pod \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") "
Jul 06 15:59:23.957125 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957013    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrmlj\" (UniqueName: \"kubernetes.io/projected/4a2f47d0-a94f-4eac-80cd-1327f82b827f-kube-api-access-qrmlj\") pod \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") "
Jul 06 15:59:23.957125 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957055    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-home\") pod \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") "
Jul 06 15:59:23.957349 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-model-cache\") pod \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\" (UID: \"4a2f47d0-a94f-4eac-80cd-1327f82b827f\") "
Jul 06 15:59:23.957349 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957298    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a2f47d0-a94f-4eac-80cd-1327f82b827f" (UID: "4a2f47d0-a94f-4eac-80cd-1327f82b827f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:59:23.957458 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957415    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-home" (OuterVolumeSpecName: "home") pod "4a2f47d0-a94f-4eac-80cd-1327f82b827f" (UID: "4a2f47d0-a94f-4eac-80cd-1327f82b827f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:59:23.957504 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957482    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a2f47d0-a94f-4eac-80cd-1327f82b827f" (UID: "4a2f47d0-a94f-4eac-80cd-1327f82b827f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:59:23.957588 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.957561    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:59:23.959392 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.959370    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-dshm" (OuterVolumeSpecName: "dshm") pod "4a2f47d0-a94f-4eac-80cd-1327f82b827f" (UID: "4a2f47d0-a94f-4eac-80cd-1327f82b827f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 15:59:23.959499 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.959426    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a2f47d0-a94f-4eac-80cd-1327f82b827f" (UID: "4a2f47d0-a94f-4eac-80cd-1327f82b827f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 15:59:23.959540 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:23.959501    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a2f47d0-a94f-4eac-80cd-1327f82b827f-kube-api-access-qrmlj" (OuterVolumeSpecName: "kube-api-access-qrmlj") pod "4a2f47d0-a94f-4eac-80cd-1327f82b827f" (UID: "4a2f47d0-a94f-4eac-80cd-1327f82b827f"). InnerVolumeSpecName "kube-api-access-qrmlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 15:59:24.058597 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.058561    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a2f47d0-a94f-4eac-80cd-1327f82b827f-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:59:24.058597 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.058593    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:59:24.058597 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.058602    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrmlj\" (UniqueName: \"kubernetes.io/projected/4a2f47d0-a94f-4eac-80cd-1327f82b827f-kube-api-access-qrmlj\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:59:24.058856 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.058611    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:59:24.058856 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.058620    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a2f47d0-a94f-4eac-80cd-1327f82b827f-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 15:59:24.248560 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.248525    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"]
Jul 06 15:59:24.251089 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:24.251065    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-37068508/e2e-lifecycle-model-37068508-kserve-6678646775-98rqj"]
Jul 06 15:59:26.022422 ip-10-0-141-57 kubenswrapper[2580]: I0706 15:59:26.022377    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a2f47d0-a94f-4eac-80cd-1327f82b827f" path="/var/lib/kubelet/pods/4a2f47d0-a94f-4eac-80cd-1327f82b827f/volumes"
Jul 06 16:01:30.997217 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:30.997177    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"]
Jul 06 16:01:30.997882 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:30.997862    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a2f47d0-a94f-4eac-80cd-1327f82b827f" containerName="main"
Jul 06 16:01:30.997967 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:30.997885    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a2f47d0-a94f-4eac-80cd-1327f82b827f" containerName="main"
Jul 06 16:01:30.998028 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:30.998013    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a2f47d0-a94f-4eac-80cd-1327f82b827f" containerName="main"
Jul 06 16:01:31.001294 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.001264    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.003488 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.003463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-cff200\"/\"openshift-service-ca.crt\""
Jul 06 16:01:31.004186 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.004169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-cff200\"/\"test-model-cff200-kserve-self-signed-certs\""
Jul 06 16:01:31.004186 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.004179    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-cff200\"/\"kube-root-ca.crt\""
Jul 06 16:01:31.004352 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.004224    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-cff200\"/\"default-dockercfg-xrdn8\""
Jul 06 16:01:31.015078 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.015041    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"]
Jul 06 16:01:31.153274 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.153227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-home\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.153274 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.153272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tmp-dir\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.153544 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.153310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-dshm\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.153544 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.153346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tls-certs\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.153544 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.153395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cjrc\" (UniqueName: \"kubernetes.io/projected/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-kube-api-access-5cjrc\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.153544 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.153440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-model-cache\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254395 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-home\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254395 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tmp-dir\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254395 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-dshm\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254725 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tls-certs\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254725 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjrc\" (UniqueName: \"kubernetes.io/projected/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-kube-api-access-5cjrc\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254725 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-model-cache\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.254942 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254912    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tmp-dir\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.255018 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.254941    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-home\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.255112 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.255088    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-model-cache\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.257024 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.256998    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-dshm\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.257260 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.257244    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tls-certs\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.263619 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.263592    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cjrc\" (UniqueName: \"kubernetes.io/projected/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-kube-api-access-5cjrc\") pod \"test-model-cff200-kserve-76cdfb675f-srr5p\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") " pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.311691 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.311640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:31.457621 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.457587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"]
Jul 06 16:01:31.459827 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:01:31.459801    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00296ffe_3e1c_43c6_a45d_87a9f03af8ed.slice/crio-6149964b5e7bf65d3fed59212847ca6d853842d4731a59a03abc3acc136182d4 WatchSource:0}: Error finding container 6149964b5e7bf65d3fed59212847ca6d853842d4731a59a03abc3acc136182d4: Status 404 returned error can't find the container with id 6149964b5e7bf65d3fed59212847ca6d853842d4731a59a03abc3acc136182d4
Jul 06 16:01:31.640753 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.640682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p" event={"ID":"00296ffe-3e1c-43c6-a45d-87a9f03af8ed","Type":"ContainerStarted","Data":"a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836"}
Jul 06 16:01:31.640753 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.640751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p" event={"ID":"00296ffe-3e1c-43c6-a45d-87a9f03af8ed","Type":"ContainerStarted","Data":"6149964b5e7bf65d3fed59212847ca6d853842d4731a59a03abc3acc136182d4"}
Jul 06 16:01:31.661282 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:31.661220    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p" podStartSLOduration=1.661200419 podStartE2EDuration="1.661200419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:01:31.658348659 +0000 UTC m=+2083.471211440" watchObservedRunningTime="2026-07-06 16:01:31.661200419 +0000 UTC m=+2083.474063196"
Jul 06 16:01:41.312491 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:41.312446    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:41.312945 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:41.312538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:41.325332 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:41.325303    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:41.711826 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:41.711795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:01:50.298748 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:50.298595    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 16:01:50.313397 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:50.304452    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 16:01:59.175972 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.175929    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"]
Jul 06 16:01:59.179484 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.179463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.182894 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.182870    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-57aaf3\"/\"default-dockercfg-m6vr7\""
Jul 06 16:01:59.183058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.182870    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-57aaf3\"/\"openshift-service-ca.crt\""
Jul 06 16:01:59.183058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.182940    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-57aaf3\"/\"test-model-57aaf3-kserve-self-signed-certs\""
Jul 06 16:01:59.183058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.182907    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-57aaf3\"/\"kube-root-ca.crt\""
Jul 06 16:01:59.188751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.188724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"]
Jul 06 16:01:59.224445 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.224399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-home\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.224907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.224461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rddq\" (UniqueName: \"kubernetes.io/projected/c574830c-22c8-49c2-9f21-b4273586897b-kube-api-access-8rddq\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.224907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.224493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-model-cache\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.224907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.224567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-dshm\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.224907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.224596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-tmp-dir\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.224907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.224691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c574830c-22c8-49c2-9f21-b4273586897b-tls-certs\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.325883 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.325844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-dshm\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326092 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.325890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-tmp-dir\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326092 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.325963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c574830c-22c8-49c2-9f21-b4273586897b-tls-certs\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326092 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.326019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-home\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326092 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.326061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rddq\" (UniqueName: \"kubernetes.io/projected/c574830c-22c8-49c2-9f21-b4273586897b-kube-api-access-8rddq\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326092 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.326089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-model-cache\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326392 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.326368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-tmp-dir\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326462 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.326409    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-home\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.326462 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.326454    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-model-cache\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.328285 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.328264    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-dshm\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.328459 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.328443    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c574830c-22c8-49c2-9f21-b4273586897b-tls-certs\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.333827 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.333796    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rddq\" (UniqueName: \"kubernetes.io/projected/c574830c-22c8-49c2-9f21-b4273586897b-kube-api-access-8rddq\") pod \"test-model-57aaf3-kserve-845f994699-nh8q8\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") " pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.490095 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.490003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:01:59.640555 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.640522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"]
Jul 06 16:01:59.642104 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:01:59.642074    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc574830c_22c8_49c2_9f21_b4273586897b.slice/crio-6863b2f987ca71e17d0cb3332a688485168df76b9c932eafb82b1e14937a2451 WatchSource:0}: Error finding container 6863b2f987ca71e17d0cb3332a688485168df76b9c932eafb82b1e14937a2451: Status 404 returned error can't find the container with id 6863b2f987ca71e17d0cb3332a688485168df76b9c932eafb82b1e14937a2451
Jul 06 16:01:59.803840 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.803742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8" event={"ID":"c574830c-22c8-49c2-9f21-b4273586897b","Type":"ContainerStarted","Data":"5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631"}
Jul 06 16:01:59.803840 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.803790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8" event={"ID":"c574830c-22c8-49c2-9f21-b4273586897b","Type":"ContainerStarted","Data":"6863b2f987ca71e17d0cb3332a688485168df76b9c932eafb82b1e14937a2451"}
Jul 06 16:01:59.825202 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:01:59.825139    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8" podStartSLOduration=0.825120256 podStartE2EDuration="825.120256ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:01:59.822622416 +0000 UTC m=+2111.635485194" watchObservedRunningTime="2026-07-06 16:01:59.825120256 +0000 UTC m=+2111.637983033"
Jul 06 16:02:09.490469 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:09.490420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:02:09.491054 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:09.490497    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:02:09.503310 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:09.503283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:02:09.871904 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:09.871874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:02:20.051363 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:20.051312    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"]
Jul 06 16:02:21.085395 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:21.085357    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"]
Jul 06 16:02:35.072754 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.072684    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p" podUID="00296ffe-3e1c-43c6-a45d-87a9f03af8ed" containerName="main" containerID="cri-o://a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836" gracePeriod=45
Jul 06 16:02:35.329432 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.329361    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:02:35.371485 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371449    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-home\") pod \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") "
Jul 06 16:02:35.371485 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cjrc\" (UniqueName: \"kubernetes.io/projected/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-kube-api-access-5cjrc\") pod \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") "
Jul 06 16:02:35.371751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371544    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-dshm\") pod \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") "
Jul 06 16:02:35.371751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tls-certs\") pod \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") "
Jul 06 16:02:35.371751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tmp-dir\") pod \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") "
Jul 06 16:02:35.371751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371676    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-model-cache\") pod \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\" (UID: \"00296ffe-3e1c-43c6-a45d-87a9f03af8ed\") "
Jul 06 16:02:35.371976 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371764    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-home" (OuterVolumeSpecName: "home") pod "00296ffe-3e1c-43c6-a45d-87a9f03af8ed" (UID: "00296ffe-3e1c-43c6-a45d-87a9f03af8ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:35.371976 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371944    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00296ffe-3e1c-43c6-a45d-87a9f03af8ed" (UID: "00296ffe-3e1c-43c6-a45d-87a9f03af8ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:35.372089 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.371994    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "00296ffe-3e1c-43c6-a45d-87a9f03af8ed" (UID: "00296ffe-3e1c-43c6-a45d-87a9f03af8ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:35.372089 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.372058    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:35.372089 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.372071    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:35.374517 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.374488    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-kube-api-access-5cjrc" (OuterVolumeSpecName: "kube-api-access-5cjrc") pod "00296ffe-3e1c-43c6-a45d-87a9f03af8ed" (UID: "00296ffe-3e1c-43c6-a45d-87a9f03af8ed"). InnerVolumeSpecName "kube-api-access-5cjrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:02:35.374517 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.374510    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-dshm" (OuterVolumeSpecName: "dshm") pod "00296ffe-3e1c-43c6-a45d-87a9f03af8ed" (UID: "00296ffe-3e1c-43c6-a45d-87a9f03af8ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:35.374677 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.374487    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00296ffe-3e1c-43c6-a45d-87a9f03af8ed" (UID: "00296ffe-3e1c-43c6-a45d-87a9f03af8ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:02:35.472660 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.472622    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:35.472660 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.472654    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:35.472660 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.472666    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:35.472939 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:35.472675    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cjrc\" (UniqueName: \"kubernetes.io/projected/00296ffe-3e1c-43c6-a45d-87a9f03af8ed-kube-api-access-5cjrc\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:36.010545 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.010511    2580 generic.go:356] "Generic (PLEG): container finished" podID="00296ffe-3e1c-43c6-a45d-87a9f03af8ed" containerID="a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836" exitCode=0
Jul 06 16:02:36.010764 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.010551    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p" event={"ID":"00296ffe-3e1c-43c6-a45d-87a9f03af8ed","Type":"ContainerDied","Data":"a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836"}
Jul 06 16:02:36.010764 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.010557    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"
Jul 06 16:02:36.010764 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.010578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p" event={"ID":"00296ffe-3e1c-43c6-a45d-87a9f03af8ed","Type":"ContainerDied","Data":"6149964b5e7bf65d3fed59212847ca6d853842d4731a59a03abc3acc136182d4"}
Jul 06 16:02:36.010764 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.010598    2580 scope.go:117] "RemoveContainer" containerID="a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836"
Jul 06 16:02:36.021771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.021737    2580 scope.go:117] "RemoveContainer" containerID="a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836"
Jul 06 16:02:36.022102 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:02:36.022071    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836\": container with ID starting with a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836 not found: ID does not exist" containerID="a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836"
Jul 06 16:02:36.022209 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.022115    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836"} err="failed to get container status \"a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836\": rpc error: code = NotFound desc = could not find container \"a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836\": container with ID starting with a740279cd3f23d9bb4579304efe9bffbaec75be1da62251c6424ae7bd2585836 not found: ID does not exist"
Jul 06 16:02:36.041342 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.041194    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"]
Jul 06 16:02:36.043944 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.043911    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-cff200/test-model-cff200-kserve-76cdfb675f-srr5p"]
Jul 06 16:02:36.107466 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.107426    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8" podUID="c574830c-22c8-49c2-9f21-b4273586897b" containerName="main" containerID="cri-o://5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631" gracePeriod=45
Jul 06 16:02:36.367745 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.367714    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:02:36.483777 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.483738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-tmp-dir\") pod \"c574830c-22c8-49c2-9f21-b4273586897b\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") "
Jul 06 16:02:36.483987 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.483809    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-home\") pod \"c574830c-22c8-49c2-9f21-b4273586897b\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") "
Jul 06 16:02:36.483987 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.483871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-model-cache\") pod \"c574830c-22c8-49c2-9f21-b4273586897b\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") "
Jul 06 16:02:36.483987 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.483948    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rddq\" (UniqueName: \"kubernetes.io/projected/c574830c-22c8-49c2-9f21-b4273586897b-kube-api-access-8rddq\") pod \"c574830c-22c8-49c2-9f21-b4273586897b\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") "
Jul 06 16:02:36.483987 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.483981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-dshm\") pod \"c574830c-22c8-49c2-9f21-b4273586897b\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") "
Jul 06 16:02:36.484193 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484005    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c574830c-22c8-49c2-9f21-b4273586897b-tls-certs\") pod \"c574830c-22c8-49c2-9f21-b4273586897b\" (UID: \"c574830c-22c8-49c2-9f21-b4273586897b\") "
Jul 06 16:02:36.484193 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484037    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c574830c-22c8-49c2-9f21-b4273586897b" (UID: "c574830c-22c8-49c2-9f21-b4273586897b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:36.484193 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484086    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-home" (OuterVolumeSpecName: "home") pod "c574830c-22c8-49c2-9f21-b4273586897b" (UID: "c574830c-22c8-49c2-9f21-b4273586897b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:36.484193 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484151    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-model-cache" (OuterVolumeSpecName: "model-cache") pod "c574830c-22c8-49c2-9f21-b4273586897b" (UID: "c574830c-22c8-49c2-9f21-b4273586897b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:36.484404 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484325    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:36.484404 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484339    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:36.484404 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.484352    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:36.486361 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.486324    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-dshm" (OuterVolumeSpecName: "dshm") pod "c574830c-22c8-49c2-9f21-b4273586897b" (UID: "c574830c-22c8-49c2-9f21-b4273586897b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:02:36.486483 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.486388    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c574830c-22c8-49c2-9f21-b4273586897b-kube-api-access-8rddq" (OuterVolumeSpecName: "kube-api-access-8rddq") pod "c574830c-22c8-49c2-9f21-b4273586897b" (UID: "c574830c-22c8-49c2-9f21-b4273586897b"). InnerVolumeSpecName "kube-api-access-8rddq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:02:36.486483 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.486397    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c574830c-22c8-49c2-9f21-b4273586897b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c574830c-22c8-49c2-9f21-b4273586897b" (UID: "c574830c-22c8-49c2-9f21-b4273586897b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:02:36.584883 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.584785    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rddq\" (UniqueName: \"kubernetes.io/projected/c574830c-22c8-49c2-9f21-b4273586897b-kube-api-access-8rddq\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:36.584883 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.584819    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c574830c-22c8-49c2-9f21-b4273586897b-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:36.584883 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:36.584833    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c574830c-22c8-49c2-9f21-b4273586897b-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:02:37.018139 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.018104    2580 generic.go:356] "Generic (PLEG): container finished" podID="c574830c-22c8-49c2-9f21-b4273586897b" containerID="5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631" exitCode=0
Jul 06 16:02:37.018139 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.018144    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"
Jul 06 16:02:37.018367 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.018181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8" event={"ID":"c574830c-22c8-49c2-9f21-b4273586897b","Type":"ContainerDied","Data":"5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631"}
Jul 06 16:02:37.018367 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.018220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8" event={"ID":"c574830c-22c8-49c2-9f21-b4273586897b","Type":"ContainerDied","Data":"6863b2f987ca71e17d0cb3332a688485168df76b9c932eafb82b1e14937a2451"}
Jul 06 16:02:37.018367 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.018238    2580 scope.go:117] "RemoveContainer" containerID="5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631"
Jul 06 16:02:37.028351 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.028328    2580 scope.go:117] "RemoveContainer" containerID="5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631"
Jul 06 16:02:37.028691 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:02:37.028669    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631\": container with ID starting with 5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631 not found: ID does not exist" containerID="5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631"
Jul 06 16:02:37.028763 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.028720    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631"} err="failed to get container status \"5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631\": rpc error: code = NotFound desc = could not find container \"5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631\": container with ID starting with 5ccb831169b394ae75fba9f58ba351621484c8cb2821047265cd7afa9c175631 not found: ID does not exist"
Jul 06 16:02:37.045565 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.045524    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"]
Jul 06 16:02:37.047898 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:37.047871    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-57aaf3/test-model-57aaf3-kserve-845f994699-nh8q8"]
Jul 06 16:02:38.022150 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:38.022113    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00296ffe-3e1c-43c6-a45d-87a9f03af8ed" path="/var/lib/kubelet/pods/00296ffe-3e1c-43c6-a45d-87a9f03af8ed/volumes"
Jul 06 16:02:38.022835 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:02:38.022814    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c574830c-22c8-49c2-9f21-b4273586897b" path="/var/lib/kubelet/pods/c574830c-22c8-49c2-9f21-b4273586897b/volumes"
Jul 06 16:04:15.697604 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.697568    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"]
Jul 06 16:04:15.698130 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.697993    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c574830c-22c8-49c2-9f21-b4273586897b" containerName="main"
Jul 06 16:04:15.698130 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.698010    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c574830c-22c8-49c2-9f21-b4273586897b" containerName="main"
Jul 06 16:04:15.698130 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.698040    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00296ffe-3e1c-43c6-a45d-87a9f03af8ed" containerName="main"
Jul 06 16:04:15.698130 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.698050    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="00296ffe-3e1c-43c6-a45d-87a9f03af8ed" containerName="main"
Jul 06 16:04:15.698269 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.698156    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="00296ffe-3e1c-43c6-a45d-87a9f03af8ed" containerName="main"
Jul 06 16:04:15.698269 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.698169    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c574830c-22c8-49c2-9f21-b4273586897b" containerName="main"
Jul 06 16:04:15.701502 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.701482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.704193 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.704168    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b75322\"/\"kube-root-ca.crt\""
Jul 06 16:04:15.704937 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.704910    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75322\"/\"auth-test-model-b75322-kserve-self-signed-certs\""
Jul 06 16:04:15.704937 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.704933    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75322\"/\"default-dockercfg-lk8dk\""
Jul 06 16:04:15.705098 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.704951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b75322\"/\"openshift-service-ca.crt\""
Jul 06 16:04:15.712076 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.712040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"]
Jul 06 16:04:15.808858 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.808816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-home\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.808858 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.808859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddz8q\" (UniqueName: \"kubernetes.io/projected/57b4250d-15c9-4301-9c4e-b2545d9c64a1-kube-api-access-ddz8q\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.809090 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.808974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-dshm\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.809090 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.809018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tls-certs\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.809090 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.809039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tmp-dir\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.809196 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.809132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-model-cache\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.910634 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.910599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-model-cache\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.910634 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.910640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-home\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.910983 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.910671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddz8q\" (UniqueName: \"kubernetes.io/projected/57b4250d-15c9-4301-9c4e-b2545d9c64a1-kube-api-access-ddz8q\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.910983 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.910780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-dshm\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.910983 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.910823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tls-certs\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.910983 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.910847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tmp-dir\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.911198 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.911143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-home\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.911243 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.911206    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tmp-dir\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.911401 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.911383    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-model-cache\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.913185 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.913161    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-dshm\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.913474 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.913451    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tls-certs\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:15.920464 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:15.920433    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddz8q\" (UniqueName: \"kubernetes.io/projected/57b4250d-15c9-4301-9c4e-b2545d9c64a1-kube-api-access-ddz8q\") pod \"auth-test-model-b75322-kserve-7d8cb94d67-64l8v\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") " pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:16.013339 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:16.013237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:16.165436 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:16.165399    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"]
Jul 06 16:04:16.168528 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:04:16.168501    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57b4250d_15c9_4301_9c4e_b2545d9c64a1.slice/crio-fb448f1b5113f59b1497d80d86ee7a99a25cf99d0ff2bd9416c34ac491bd9aa8 WatchSource:0}: Error finding container fb448f1b5113f59b1497d80d86ee7a99a25cf99d0ff2bd9416c34ac491bd9aa8: Status 404 returned error can't find the container with id fb448f1b5113f59b1497d80d86ee7a99a25cf99d0ff2bd9416c34ac491bd9aa8
Jul 06 16:04:16.583221 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:16.583189    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v" event={"ID":"57b4250d-15c9-4301-9c4e-b2545d9c64a1","Type":"ContainerStarted","Data":"abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7"}
Jul 06 16:04:16.583413 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:16.583234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v" event={"ID":"57b4250d-15c9-4301-9c4e-b2545d9c64a1","Type":"ContainerStarted","Data":"fb448f1b5113f59b1497d80d86ee7a99a25cf99d0ff2bd9416c34ac491bd9aa8"}
Jul 06 16:04:16.600904 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:16.600843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v" podStartSLOduration=1.600821541 podStartE2EDuration="1.600821541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:04:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:04:16.600169885 +0000 UTC m=+2248.413032662" watchObservedRunningTime="2026-07-06 16:04:16.600821541 +0000 UTC m=+2248.413684319"
Jul 06 16:04:26.014192 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:26.014147    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:26.014192 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:26.014208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:26.027112 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:26.027080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:26.649825 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:26.649789    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:04:45.312157 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.312113    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"]
Jul 06 16:04:45.315856 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.315829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.320357 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.320334    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"kube-root-ca.crt\""
Jul 06 16:04:45.320357 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.320352    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"openshift-service-ca.crt\""
Jul 06 16:04:45.320544 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.320332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"default-dockercfg-9t4c5\""
Jul 06 16:04:45.320544 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.320330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"auth-test-model-690f40-kserve-self-signed-certs\""
Jul 06 16:04:45.391885 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.391836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tmp-dir\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.391885 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.391884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tls-certs\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.392109 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.392027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-dshm\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.392109 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.392064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nnfr\" (UniqueName: \"kubernetes.io/projected/41369d50-3c95-4b86-bd8c-2ff53f044bb4-kube-api-access-8nnfr\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.392179 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.392125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-home\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.392179 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.392159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-model-cache\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.429938 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.429898    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"]
Jul 06 16:04:45.492839 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.492795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-home\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.492851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-model-cache\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.492888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tmp-dir\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.492911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tls-certs\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.493005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-dshm\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493058 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.493026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nnfr\" (UniqueName: \"kubernetes.io/projected/41369d50-3c95-4b86-bd8c-2ff53f044bb4-kube-api-access-8nnfr\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493312 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.493288    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-model-cache\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493312 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.493303    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-home\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.493409 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.493354    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tmp-dir\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.495379 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.495349    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-dshm\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.495616 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.495597    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tls-certs\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.507721 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.507671    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nnfr\" (UniqueName: \"kubernetes.io/projected/41369d50-3c95-4b86-bd8c-2ff53f044bb4-kube-api-access-8nnfr\") pod \"auth-test-model-690f40-kserve-7bbbb7f575-gncwx\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") " pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.626864 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.626829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:45.775343 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:45.775311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"]
Jul 06 16:04:45.777766 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:04:45.777739    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41369d50_3c95_4b86_bd8c_2ff53f044bb4.slice/crio-53ca45172edfb735d7639611aaa9d2e84a952033e87fa7953174373f95e5ab45 WatchSource:0}: Error finding container 53ca45172edfb735d7639611aaa9d2e84a952033e87fa7953174373f95e5ab45: Status 404 returned error can't find the container with id 53ca45172edfb735d7639611aaa9d2e84a952033e87fa7953174373f95e5ab45
Jul 06 16:04:46.762482 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:46.762442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" event={"ID":"41369d50-3c95-4b86-bd8c-2ff53f044bb4","Type":"ContainerStarted","Data":"cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d"}
Jul 06 16:04:46.762982 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:46.762495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" event={"ID":"41369d50-3c95-4b86-bd8c-2ff53f044bb4","Type":"ContainerStarted","Data":"53ca45172edfb735d7639611aaa9d2e84a952033e87fa7953174373f95e5ab45"}
Jul 06 16:04:46.780757 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:46.780670    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" podStartSLOduration=1.780649996 podStartE2EDuration="1.780649996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:04:46.779211578 +0000 UTC m=+2278.592074356" watchObservedRunningTime="2026-07-06 16:04:46.780649996 +0000 UTC m=+2278.593512772"
Jul 06 16:04:55.627168 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:55.627122    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:55.627733 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:55.627188    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:55.639923 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:55.639894    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:04:55.823031 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:04:55.822996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:06:39.656565 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:39.656529    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"]
Jul 06 16:06:40.546631 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:40.546588    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"]
Jul 06 16:06:44.773823 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.773784    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"]
Jul 06 16:06:44.777233 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.777211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.779515 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.779488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75322\"/\"sub-test-model-b75322-kserve-self-signed-certs\""
Jul 06 16:06:44.789588 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.789545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"]
Jul 06 16:06:44.844354 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.844312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-dshm\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.844552 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.844370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-home\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.844552 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.844431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57snm\" (UniqueName: \"kubernetes.io/projected/83df852c-9e02-446a-b227-56e380f9a328-kube-api-access-57snm\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.844552 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.844470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-model-cache\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.844552 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.844489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-tmp-dir\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.844552 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.844520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83df852c-9e02-446a-b227-56e380f9a328-tls-certs\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.945590 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.945552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-dshm\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.945810 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.945602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-home\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.945810 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.945757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57snm\" (UniqueName: \"kubernetes.io/projected/83df852c-9e02-446a-b227-56e380f9a328-kube-api-access-57snm\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.945922 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.945807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-model-cache\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.945922 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.945838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-tmp-dir\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.945922 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.945887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83df852c-9e02-446a-b227-56e380f9a328-tls-certs\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.946170 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.946107    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-home\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.946234 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.946209    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-model-cache\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.946380 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.946357    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-tmp-dir\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.947966 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.947946    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-dshm\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.948392 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.948369    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83df852c-9e02-446a-b227-56e380f9a328-tls-certs\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:44.954606 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:44.954575    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57snm\" (UniqueName: \"kubernetes.io/projected/83df852c-9e02-446a-b227-56e380f9a328-kube-api-access-57snm\") pod \"sub-test-model-b75322-kserve-85d886f4f7-zk48b\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") " pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:45.086830 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:45.086742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:45.242142 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:45.242111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"]
Jul 06 16:06:45.244317 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:06:45.244282    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83df852c_9e02_446a_b227_56e380f9a328.slice/crio-d5ca4d8ed913b6c75346874b2e51d493253c23ada63b69c5e70543f7b5827a30 WatchSource:0}: Error finding container d5ca4d8ed913b6c75346874b2e51d493253c23ada63b69c5e70543f7b5827a30: Status 404 returned error can't find the container with id d5ca4d8ed913b6c75346874b2e51d493253c23ada63b69c5e70543f7b5827a30
Jul 06 16:06:45.450015 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:45.449979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b" event={"ID":"83df852c-9e02-446a-b227-56e380f9a328","Type":"ContainerStarted","Data":"0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972"}
Jul 06 16:06:45.450202 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:45.450024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b" event={"ID":"83df852c-9e02-446a-b227-56e380f9a328","Type":"ContainerStarted","Data":"d5ca4d8ed913b6c75346874b2e51d493253c23ada63b69c5e70543f7b5827a30"}
Jul 06 16:06:45.467888 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:45.467827    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b" podStartSLOduration=1.467806223 podStartE2EDuration="1.467806223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:06:45.466467283 +0000 UTC m=+2397.279330062" watchObservedRunningTime="2026-07-06 16:06:45.467806223 +0000 UTC m=+2397.280669040"
Jul 06 16:06:50.338777 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:50.338624    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 16:06:50.348976 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:50.345595    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-52rsz_0eabb18f-d646-4167-ad91-f6d61108bc5c/ovn-acl-logging/0.log"
Jul 06 16:06:54.677976 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:54.677926    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v" podUID="57b4250d-15c9-4301-9c4e-b2545d9c64a1" containerName="main" containerID="cri-o://abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7" gracePeriod=45
Jul 06 16:06:54.944661 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:54.944586    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:06:55.049709 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.049667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tmp-dir\") pod \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") "
Jul 06 16:06:55.049925 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.049750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-model-cache\") pod \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") "
Jul 06 16:06:55.049925 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.049789    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddz8q\" (UniqueName: \"kubernetes.io/projected/57b4250d-15c9-4301-9c4e-b2545d9c64a1-kube-api-access-ddz8q\") pod \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") "
Jul 06 16:06:55.049925 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.049816    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tls-certs\") pod \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") "
Jul 06 16:06:55.049925 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.049839    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-dshm\") pod \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") "
Jul 06 16:06:55.049925 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.049861    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-home\") pod \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\" (UID: \"57b4250d-15c9-4301-9c4e-b2545d9c64a1\") "
Jul 06 16:06:55.050172 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.050064    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "57b4250d-15c9-4301-9c4e-b2545d9c64a1" (UID: "57b4250d-15c9-4301-9c4e-b2545d9c64a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.050172 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.050076    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "57b4250d-15c9-4301-9c4e-b2545d9c64a1" (UID: "57b4250d-15c9-4301-9c4e-b2545d9c64a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.050245 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.050205    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-home" (OuterVolumeSpecName: "home") pod "57b4250d-15c9-4301-9c4e-b2545d9c64a1" (UID: "57b4250d-15c9-4301-9c4e-b2545d9c64a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.052227 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.052192    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "57b4250d-15c9-4301-9c4e-b2545d9c64a1" (UID: "57b4250d-15c9-4301-9c4e-b2545d9c64a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:06:55.052227 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.052206    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b4250d-15c9-4301-9c4e-b2545d9c64a1-kube-api-access-ddz8q" (OuterVolumeSpecName: "kube-api-access-ddz8q") pod "57b4250d-15c9-4301-9c4e-b2545d9c64a1" (UID: "57b4250d-15c9-4301-9c4e-b2545d9c64a1"). InnerVolumeSpecName "kube-api-access-ddz8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:06:55.052381 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.052228    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-dshm" (OuterVolumeSpecName: "dshm") pod "57b4250d-15c9-4301-9c4e-b2545d9c64a1" (UID: "57b4250d-15c9-4301-9c4e-b2545d9c64a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.087569 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.087527    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:55.087816 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.087610    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:55.100435 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.100402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:55.150631 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.150591    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.150631 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.150622    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.150631 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.150632    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddz8q\" (UniqueName: \"kubernetes.io/projected/57b4250d-15c9-4301-9c4e-b2545d9c64a1-kube-api-access-ddz8q\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.150631 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.150641    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57b4250d-15c9-4301-9c4e-b2545d9c64a1-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.150962 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.150649    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.150962 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.150657    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57b4250d-15c9-4301-9c4e-b2545d9c64a1-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.508426 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.508393    2580 generic.go:356] "Generic (PLEG): container finished" podID="57b4250d-15c9-4301-9c4e-b2545d9c64a1" containerID="abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7" exitCode=0
Jul 06 16:06:55.508619 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.508439    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"
Jul 06 16:06:55.508619 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.508483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v" event={"ID":"57b4250d-15c9-4301-9c4e-b2545d9c64a1","Type":"ContainerDied","Data":"abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7"}
Jul 06 16:06:55.508619 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.508530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v" event={"ID":"57b4250d-15c9-4301-9c4e-b2545d9c64a1","Type":"ContainerDied","Data":"fb448f1b5113f59b1497d80d86ee7a99a25cf99d0ff2bd9416c34ac491bd9aa8"}
Jul 06 16:06:55.508619 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.508552    2580 scope.go:117] "RemoveContainer" containerID="abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7"
Jul 06 16:06:55.519109 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.519090    2580 scope.go:117] "RemoveContainer" containerID="abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7"
Jul 06 16:06:55.519446 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:06:55.519421    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7\": container with ID starting with abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7 not found: ID does not exist" containerID="abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7"
Jul 06 16:06:55.519508 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.519456    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7"} err="failed to get container status \"abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7\": rpc error: code = NotFound desc = could not find container \"abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7\": container with ID starting with abee4e773831c5ec77b21d6d9c20e7b0e5e04fb256ae8a1bb42a9eba4b334aa7 not found: ID does not exist"
Jul 06 16:06:55.521303 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.521278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:06:55.534243 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.534179    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"]
Jul 06 16:06:55.537219 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.537187    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/auth-test-model-b75322-kserve-7d8cb94d67-64l8v"]
Jul 06 16:06:55.568242 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.568197    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" podUID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" containerName="main" containerID="cri-o://cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d" gracePeriod=45
Jul 06 16:06:55.827737 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.827684    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:06:55.958432 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nnfr\" (UniqueName: \"kubernetes.io/projected/41369d50-3c95-4b86-bd8c-2ff53f044bb4-kube-api-access-8nnfr\") pod \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") "
Jul 06 16:06:55.958620 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-home\") pod \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") "
Jul 06 16:06:55.958620 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958538    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-dshm\") pod \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") "
Jul 06 16:06:55.958620 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958561    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-model-cache\") pod \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") "
Jul 06 16:06:55.958788 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tls-certs\") pod \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") "
Jul 06 16:06:55.958788 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958690    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tmp-dir\") pod \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\" (UID: \"41369d50-3c95-4b86-bd8c-2ff53f044bb4\") "
Jul 06 16:06:55.958899 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958823    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-home" (OuterVolumeSpecName: "home") pod "41369d50-3c95-4b86-bd8c-2ff53f044bb4" (UID: "41369d50-3c95-4b86-bd8c-2ff53f044bb4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.958957 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958902    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-model-cache" (OuterVolumeSpecName: "model-cache") pod "41369d50-3c95-4b86-bd8c-2ff53f044bb4" (UID: "41369d50-3c95-4b86-bd8c-2ff53f044bb4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.959005 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.958983    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41369d50-3c95-4b86-bd8c-2ff53f044bb4" (UID: "41369d50-3c95-4b86-bd8c-2ff53f044bb4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.959171 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.959151    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.959231 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.959176    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.959231 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.959186    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:55.960884 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.960860    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-dshm" (OuterVolumeSpecName: "dshm") pod "41369d50-3c95-4b86-bd8c-2ff53f044bb4" (UID: "41369d50-3c95-4b86-bd8c-2ff53f044bb4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:06:55.960981 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.960867    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41369d50-3c95-4b86-bd8c-2ff53f044bb4-kube-api-access-8nnfr" (OuterVolumeSpecName: "kube-api-access-8nnfr") pod "41369d50-3c95-4b86-bd8c-2ff53f044bb4" (UID: "41369d50-3c95-4b86-bd8c-2ff53f044bb4"). InnerVolumeSpecName "kube-api-access-8nnfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:06:55.960981 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:55.960912    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41369d50-3c95-4b86-bd8c-2ff53f044bb4" (UID: "41369d50-3c95-4b86-bd8c-2ff53f044bb4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:06:56.022901 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.022860    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57b4250d-15c9-4301-9c4e-b2545d9c64a1" path="/var/lib/kubelet/pods/57b4250d-15c9-4301-9c4e-b2545d9c64a1/volumes"
Jul 06 16:06:56.060587 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.060482    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41369d50-3c95-4b86-bd8c-2ff53f044bb4-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:56.060587 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.060516    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41369d50-3c95-4b86-bd8c-2ff53f044bb4-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:56.060587 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.060528    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nnfr\" (UniqueName: \"kubernetes.io/projected/41369d50-3c95-4b86-bd8c-2ff53f044bb4-kube-api-access-8nnfr\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:06:56.515917 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.515877    2580 generic.go:356] "Generic (PLEG): container finished" podID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" containerID="cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d" exitCode=0
Jul 06 16:06:56.516127 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.515925    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"
Jul 06 16:06:56.516127 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.515963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" event={"ID":"41369d50-3c95-4b86-bd8c-2ff53f044bb4","Type":"ContainerDied","Data":"cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d"}
Jul 06 16:06:56.516127 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.516011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" event={"ID":"41369d50-3c95-4b86-bd8c-2ff53f044bb4","Type":"ContainerDied","Data":"53ca45172edfb735d7639611aaa9d2e84a952033e87fa7953174373f95e5ab45"}
Jul 06 16:06:56.516127 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.516028    2580 scope.go:117] "RemoveContainer" containerID="cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d"
Jul 06 16:06:56.525546 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.525522    2580 scope.go:117] "RemoveContainer" containerID="cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d"
Jul 06 16:06:56.525882 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:06:56.525860    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d\": container with ID starting with cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d not found: ID does not exist" containerID="cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d"
Jul 06 16:06:56.525947 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.525890    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d"} err="failed to get container status \"cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d\": rpc error: code = NotFound desc = could not find container \"cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d\": container with ID starting with cfed13e5aed17db7836e7f74e1ccdba288164b21dfd2c158cd975d47a9fdd04d not found: ID does not exist"
Jul 06 16:06:56.536761 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.536728    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"]
Jul 06 16:06:56.541516 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:56.541477    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx"]
Jul 06 16:06:58.021930 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:06:58.021891    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" path="/var/lib/kubelet/pods/41369d50-3c95-4b86-bd8c-2ff53f044bb4/volumes"
Jul 06 16:07:00.812794 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:00.812744    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-690f40/auth-test-model-690f40-kserve-7bbbb7f575-gncwx" podUID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Jul 06 16:07:19.881516 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.881476    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"]
Jul 06 16:07:19.882065 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.881906    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" containerName="main"
Jul 06 16:07:19.882065 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.881920    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" containerName="main"
Jul 06 16:07:19.882065 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.881936    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57b4250d-15c9-4301-9c4e-b2545d9c64a1" containerName="main"
Jul 06 16:07:19.882065 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.881942    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57b4250d-15c9-4301-9c4e-b2545d9c64a1" containerName="main"
Jul 06 16:07:19.882065 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.882009    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="41369d50-3c95-4b86-bd8c-2ff53f044bb4" containerName="main"
Jul 06 16:07:19.882065 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.882018    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57b4250d-15c9-4301-9c4e-b2545d9c64a1" containerName="main"
Jul 06 16:07:19.886662 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.886632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:19.889119 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.889089    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"openshift-service-ca.crt\""
Jul 06 16:07:19.889820 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.889799    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"kube-root-ca.crt\""
Jul 06 16:07:19.889945 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.889799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"default-dockercfg-9t4c5\""
Jul 06 16:07:19.889945 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.889799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"sub-test-model-690f40-kserve-self-signed-certs\""
Jul 06 16:07:19.897291 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.897261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"]
Jul 06 16:07:19.998060 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.998018    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9b88\" (UniqueName: \"kubernetes.io/projected/a2b4919c-65d3-4e23-9cf7-c854a149552e-kube-api-access-j9b88\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:19.998253 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.998074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-tmp-dir\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:19.998253 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.998143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b4919c-65d3-4e23-9cf7-c854a149552e-tls-certs\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:19.998253 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.998175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-home\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:19.998253 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.998232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-model-cache\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:19.998393 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:19.998274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-dshm\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099379 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9b88\" (UniqueName: \"kubernetes.io/projected/a2b4919c-65d3-4e23-9cf7-c854a149552e-kube-api-access-j9b88\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099562 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-tmp-dir\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099562 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b4919c-65d3-4e23-9cf7-c854a149552e-tls-certs\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099562 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-home\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099562 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-model-cache\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099562 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-dshm\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099922 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099900    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-tmp-dir\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099973 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-home\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.099973 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.099956    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-model-cache\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.101932 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.101911    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-dshm\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.102183 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.102166    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b4919c-65d3-4e23-9cf7-c854a149552e-tls-certs\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.108107 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.108079    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9b88\" (UniqueName: \"kubernetes.io/projected/a2b4919c-65d3-4e23-9cf7-c854a149552e-kube-api-access-j9b88\") pod \"sub-test-model-690f40-kserve-97f9687c7-fz9fp\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") " pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.198602 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.198500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:20.345143 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.345106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"]
Jul 06 16:07:20.346952 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:07:20.346926    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2b4919c_65d3_4e23_9cf7_c854a149552e.slice/crio-b3b7914d0a8b0231f29496b9a90201b0de573dcc4626f1c9f5aa9acb1ff80d81 WatchSource:0}: Error finding container b3b7914d0a8b0231f29496b9a90201b0de573dcc4626f1c9f5aa9acb1ff80d81: Status 404 returned error can't find the container with id b3b7914d0a8b0231f29496b9a90201b0de573dcc4626f1c9f5aa9acb1ff80d81
Jul 06 16:07:20.654917 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.654876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp" event={"ID":"a2b4919c-65d3-4e23-9cf7-c854a149552e","Type":"ContainerStarted","Data":"cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67"}
Jul 06 16:07:20.655105 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.654926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp" event={"ID":"a2b4919c-65d3-4e23-9cf7-c854a149552e","Type":"ContainerStarted","Data":"b3b7914d0a8b0231f29496b9a90201b0de573dcc4626f1c9f5aa9acb1ff80d81"}
Jul 06 16:07:20.677902 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:20.677846    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp" podStartSLOduration=1.677828548 podStartE2EDuration="1.677828548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:07:20.677670169 +0000 UTC m=+2432.490532949" watchObservedRunningTime="2026-07-06 16:07:20.677828548 +0000 UTC m=+2432.490691325"
Jul 06 16:07:30.199587 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:30.199546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:30.200035 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:30.199607    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:30.212176 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:30.212148    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:07:30.724330 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:07:30.724296    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:08:05.060474 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:05.060428    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"]
Jul 06 16:08:05.994495 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:05.994451    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"]
Jul 06 16:08:13.785816 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.785775    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"]
Jul 06 16:08:13.789415 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.789395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:13.791856 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.791833    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b75322\"/\"rate-test-model-b75322-kserve-self-signed-certs\""
Jul 06 16:08:13.804586 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.804551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"]
Jul 06 16:08:13.924858 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.924800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-model-cache\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:13.925076 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.924921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tmp-dir\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:13.925076 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.924963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-home\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:13.925076 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.924988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-dshm\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:13.925076 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.925005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6h6p\" (UniqueName: \"kubernetes.io/projected/a2674f40-a3fa-4c41-aa7c-26e68b62173a-kube-api-access-q6h6p\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:13.925234 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:13.925116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tls-certs\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.026561 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.026522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-home\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.026561 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.026563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-dshm\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.026870 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.026581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6h6p\" (UniqueName: \"kubernetes.io/projected/a2674f40-a3fa-4c41-aa7c-26e68b62173a-kube-api-access-q6h6p\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.026870 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.026619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tls-certs\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.026870 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.026666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-model-cache\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.026870 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.026746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tmp-dir\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.027115 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.027086    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-home\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.027208 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.027188    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-model-cache\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.027262 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.027188    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tmp-dir\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.028910 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.028886    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-dshm\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.029280 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.029259    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tls-certs\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.035907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.035848    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6h6p\" (UniqueName: \"kubernetes.io/projected/a2674f40-a3fa-4c41-aa7c-26e68b62173a-kube-api-access-q6h6p\") pod \"rate-test-model-b75322-kserve-76465c4b5f-xqf5h\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") " pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.099128 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.099091    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:14.245645 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.245606    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"]
Jul 06 16:08:14.247493 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:08:14.247460    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2674f40_a3fa_4c41_aa7c_26e68b62173a.slice/crio-8df80310a4067a15c628870ecd168dd0878277588244237cba9f9e30c4120abe WatchSource:0}: Error finding container 8df80310a4067a15c628870ecd168dd0878277588244237cba9f9e30c4120abe: Status 404 returned error can't find the container with id 8df80310a4067a15c628870ecd168dd0878277588244237cba9f9e30c4120abe
Jul 06 16:08:14.959948 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.959908    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h" event={"ID":"a2674f40-a3fa-4c41-aa7c-26e68b62173a","Type":"ContainerStarted","Data":"1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74"}
Jul 06 16:08:14.960363 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.959957    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h" event={"ID":"a2674f40-a3fa-4c41-aa7c-26e68b62173a","Type":"ContainerStarted","Data":"8df80310a4067a15c628870ecd168dd0878277588244237cba9f9e30c4120abe"}
Jul 06 16:08:14.983645 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:14.983583    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h" podStartSLOduration=1.983567853 podStartE2EDuration="1.983567853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:08:14.982413152 +0000 UTC m=+2486.795275949" watchObservedRunningTime="2026-07-06 16:08:14.983567853 +0000 UTC m=+2486.796430631"
Jul 06 16:08:20.082089 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.082044    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b" podUID="83df852c-9e02-446a-b227-56e380f9a328" containerName="main" containerID="cri-o://0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972" gracePeriod=45
Jul 06 16:08:20.353327 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.353237    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:08:20.393547 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83df852c-9e02-446a-b227-56e380f9a328-tls-certs\") pod \"83df852c-9e02-446a-b227-56e380f9a328\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") "
Jul 06 16:08:20.393771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393553    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57snm\" (UniqueName: \"kubernetes.io/projected/83df852c-9e02-446a-b227-56e380f9a328-kube-api-access-57snm\") pod \"83df852c-9e02-446a-b227-56e380f9a328\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") "
Jul 06 16:08:20.393771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393587    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-model-cache\") pod \"83df852c-9e02-446a-b227-56e380f9a328\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") "
Jul 06 16:08:20.393771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-dshm\") pod \"83df852c-9e02-446a-b227-56e380f9a328\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") "
Jul 06 16:08:20.393771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-home\") pod \"83df852c-9e02-446a-b227-56e380f9a328\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") "
Jul 06 16:08:20.393771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-tmp-dir\") pod \"83df852c-9e02-446a-b227-56e380f9a328\" (UID: \"83df852c-9e02-446a-b227-56e380f9a328\") "
Jul 06 16:08:20.394086 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393906    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-model-cache" (OuterVolumeSpecName: "model-cache") pod "83df852c-9e02-446a-b227-56e380f9a328" (UID: "83df852c-9e02-446a-b227-56e380f9a328"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:20.394086 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.393972    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-home" (OuterVolumeSpecName: "home") pod "83df852c-9e02-446a-b227-56e380f9a328" (UID: "83df852c-9e02-446a-b227-56e380f9a328"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:20.394201 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.394089    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:20.394201 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.394111    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:20.394201 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.394087    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "83df852c-9e02-446a-b227-56e380f9a328" (UID: "83df852c-9e02-446a-b227-56e380f9a328"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:20.395876 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.395840    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-dshm" (OuterVolumeSpecName: "dshm") pod "83df852c-9e02-446a-b227-56e380f9a328" (UID: "83df852c-9e02-446a-b227-56e380f9a328"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:20.395876 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.395855    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83df852c-9e02-446a-b227-56e380f9a328-kube-api-access-57snm" (OuterVolumeSpecName: "kube-api-access-57snm") pod "83df852c-9e02-446a-b227-56e380f9a328" (UID: "83df852c-9e02-446a-b227-56e380f9a328"). InnerVolumeSpecName "kube-api-access-57snm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:08:20.396031 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.395942    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83df852c-9e02-446a-b227-56e380f9a328-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "83df852c-9e02-446a-b227-56e380f9a328" (UID: "83df852c-9e02-446a-b227-56e380f9a328"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:08:20.494569 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.494525    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83df852c-9e02-446a-b227-56e380f9a328-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:20.494569 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.494561    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57snm\" (UniqueName: \"kubernetes.io/projected/83df852c-9e02-446a-b227-56e380f9a328-kube-api-access-57snm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:20.494569 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.494574    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:20.494902 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.494590    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83df852c-9e02-446a-b227-56e380f9a328-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:20.997989 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.997953    2580 generic.go:356] "Generic (PLEG): container finished" podID="83df852c-9e02-446a-b227-56e380f9a328" containerID="0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972" exitCode=0
Jul 06 16:08:20.998155 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.998019    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"
Jul 06 16:08:20.998155 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.998033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b" event={"ID":"83df852c-9e02-446a-b227-56e380f9a328","Type":"ContainerDied","Data":"0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972"}
Jul 06 16:08:20.998155 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.998080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b" event={"ID":"83df852c-9e02-446a-b227-56e380f9a328","Type":"ContainerDied","Data":"d5ca4d8ed913b6c75346874b2e51d493253c23ada63b69c5e70543f7b5827a30"}
Jul 06 16:08:20.998155 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:20.998098    2580 scope.go:117] "RemoveContainer" containerID="0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972"
Jul 06 16:08:21.010556 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.010523    2580 scope.go:117] "RemoveContainer" containerID="0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972"
Jul 06 16:08:21.011121 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:08:21.011092    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972\": container with ID starting with 0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972 not found: ID does not exist" containerID="0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972"
Jul 06 16:08:21.011259 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.011126    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972"} err="failed to get container status \"0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972\": rpc error: code = NotFound desc = could not find container \"0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972\": container with ID starting with 0f4dc1437a9e429c497b3a9f8f1e243c718c8e0ac39fb5c8003742d2df117972 not found: ID does not exist"
Jul 06 16:08:21.014668 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.014619    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp" podUID="a2b4919c-65d3-4e23-9cf7-c854a149552e" containerName="main" containerID="cri-o://cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67" gracePeriod=45
Jul 06 16:08:21.021575 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.021543    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"]
Jul 06 16:08:21.025773 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.025743    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/sub-test-model-b75322-kserve-85d886f4f7-zk48b"]
Jul 06 16:08:21.285649 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.285621    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:08:21.301762 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.301681    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9b88\" (UniqueName: \"kubernetes.io/projected/a2b4919c-65d3-4e23-9cf7-c854a149552e-kube-api-access-j9b88\") pod \"a2b4919c-65d3-4e23-9cf7-c854a149552e\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") "
Jul 06 16:08:21.301762 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.301780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-model-cache\") pod \"a2b4919c-65d3-4e23-9cf7-c854a149552e\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") "
Jul 06 16:08:21.302028 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.301835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b4919c-65d3-4e23-9cf7-c854a149552e-tls-certs\") pod \"a2b4919c-65d3-4e23-9cf7-c854a149552e\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") "
Jul 06 16:08:21.302028 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.301860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-home\") pod \"a2b4919c-65d3-4e23-9cf7-c854a149552e\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") "
Jul 06 16:08:21.302028 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.301891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-dshm\") pod \"a2b4919c-65d3-4e23-9cf7-c854a149552e\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") "
Jul 06 16:08:21.302028 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.301936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-tmp-dir\") pod \"a2b4919c-65d3-4e23-9cf7-c854a149552e\" (UID: \"a2b4919c-65d3-4e23-9cf7-c854a149552e\") "
Jul 06 16:08:21.302240 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.302108    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2b4919c-65d3-4e23-9cf7-c854a149552e" (UID: "a2b4919c-65d3-4e23-9cf7-c854a149552e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:21.302240 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.302169    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-home" (OuterVolumeSpecName: "home") pod "a2b4919c-65d3-4e23-9cf7-c854a149552e" (UID: "a2b4919c-65d3-4e23-9cf7-c854a149552e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:21.302343 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.302315    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:21.302343 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.302334    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:21.302441 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.302343    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2b4919c-65d3-4e23-9cf7-c854a149552e" (UID: "a2b4919c-65d3-4e23-9cf7-c854a149552e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:21.304076 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.304046    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-dshm" (OuterVolumeSpecName: "dshm") pod "a2b4919c-65d3-4e23-9cf7-c854a149552e" (UID: "a2b4919c-65d3-4e23-9cf7-c854a149552e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:08:21.304197 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.304084    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b4919c-65d3-4e23-9cf7-c854a149552e-kube-api-access-j9b88" (OuterVolumeSpecName: "kube-api-access-j9b88") pod "a2b4919c-65d3-4e23-9cf7-c854a149552e" (UID: "a2b4919c-65d3-4e23-9cf7-c854a149552e"). InnerVolumeSpecName "kube-api-access-j9b88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:08:21.304355 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.304336    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2b4919c-65d3-4e23-9cf7-c854a149552e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2b4919c-65d3-4e23-9cf7-c854a149552e" (UID: "a2b4919c-65d3-4e23-9cf7-c854a149552e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:08:21.403593 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.403545    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b4919c-65d3-4e23-9cf7-c854a149552e-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:21.403593 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.403582    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:21.403593 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.403591    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b4919c-65d3-4e23-9cf7-c854a149552e-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:21.403593 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:21.403599    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9b88\" (UniqueName: \"kubernetes.io/projected/a2b4919c-65d3-4e23-9cf7-c854a149552e-kube-api-access-j9b88\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:08:22.004883 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.004841    2580 generic.go:356] "Generic (PLEG): container finished" podID="a2b4919c-65d3-4e23-9cf7-c854a149552e" containerID="cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67" exitCode=0
Jul 06 16:08:22.005093 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.004894    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"
Jul 06 16:08:22.005093 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.004929    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp" event={"ID":"a2b4919c-65d3-4e23-9cf7-c854a149552e","Type":"ContainerDied","Data":"cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67"}
Jul 06 16:08:22.005093 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.004976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp" event={"ID":"a2b4919c-65d3-4e23-9cf7-c854a149552e","Type":"ContainerDied","Data":"b3b7914d0a8b0231f29496b9a90201b0de573dcc4626f1c9f5aa9acb1ff80d81"}
Jul 06 16:08:22.005093 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.004996    2580 scope.go:117] "RemoveContainer" containerID="cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67"
Jul 06 16:08:22.014745 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.014719    2580 scope.go:117] "RemoveContainer" containerID="cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67"
Jul 06 16:08:22.015040 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:08:22.015022    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67\": container with ID starting with cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67 not found: ID does not exist" containerID="cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67"
Jul 06 16:08:22.015092 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.015048    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67"} err="failed to get container status \"cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67\": rpc error: code = NotFound desc = could not find container \"cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67\": container with ID starting with cb062bf858affbfb8fced1921de3fb1aea8995bc03888ddbffad4baef4de1a67 not found: ID does not exist"
Jul 06 16:08:22.021406 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.021364    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83df852c-9e02-446a-b227-56e380f9a328" path="/var/lib/kubelet/pods/83df852c-9e02-446a-b227-56e380f9a328/volumes"
Jul 06 16:08:22.029018 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.028974    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"]
Jul 06 16:08:22.033049 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:22.033018    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/sub-test-model-690f40-kserve-97f9687c7-fz9fp"]
Jul 06 16:08:24.022681 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:24.022629    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2b4919c-65d3-4e23-9cf7-c854a149552e" path="/var/lib/kubelet/pods/a2b4919c-65d3-4e23-9cf7-c854a149552e/volumes"
Jul 06 16:08:24.099674 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:24.099638    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:24.099837 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:24.099727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:24.112275 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:24.112247    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:25.036902 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:25.036872    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:08:43.377181 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377092    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"]
Jul 06 16:08:43.377729 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377712    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2b4919c-65d3-4e23-9cf7-c854a149552e" containerName="main"
Jul 06 16:08:43.377782 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377733    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2b4919c-65d3-4e23-9cf7-c854a149552e" containerName="main"
Jul 06 16:08:43.377782 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377753    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83df852c-9e02-446a-b227-56e380f9a328" containerName="main"
Jul 06 16:08:43.377782 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377762    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="83df852c-9e02-446a-b227-56e380f9a328" containerName="main"
Jul 06 16:08:43.377903 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377891    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="83df852c-9e02-446a-b227-56e380f9a328" containerName="main"
Jul 06 16:08:43.377943 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.377912    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2b4919c-65d3-4e23-9cf7-c854a149552e" containerName="main"
Jul 06 16:08:43.381524 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.381501    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.384495 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.384465    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"openshift-service-ca.crt\""
Jul 06 16:08:43.384495 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.384463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"rate-test-model-690f40-kserve-self-signed-certs\""
Jul 06 16:08:43.384716 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.384503    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"kube-root-ca.crt\""
Jul 06 16:08:43.384716 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.384463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-690f40\"/\"default-dockercfg-9t4c5\""
Jul 06 16:08:43.391575 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.391536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"]
Jul 06 16:08:43.517507 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.517468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-model-cache\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.517741 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.517523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rk6\" (UniqueName: \"kubernetes.io/projected/1ed76e5d-be64-49c0-adee-a971a27cc0a3-kube-api-access-94rk6\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.517741 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.517549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-dshm\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.517741 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.517626    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-home\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.517741 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.517664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tls-certs\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.517889 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.517780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tmp-dir\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619105 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tmp-dir\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619274 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-model-cache\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619274 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rk6\" (UniqueName: \"kubernetes.io/projected/1ed76e5d-be64-49c0-adee-a971a27cc0a3-kube-api-access-94rk6\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619274 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-dshm\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619447 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-home\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619447 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tls-certs\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619542 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619458    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tmp-dir\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619588 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-model-cache\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.619731 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.619715    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-home\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.621546 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.621517    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-dshm\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.621926 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.621901    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tls-certs\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.627935 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.627882    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rk6\" (UniqueName: \"kubernetes.io/projected/1ed76e5d-be64-49c0-adee-a971a27cc0a3-kube-api-access-94rk6\") pod \"rate-test-model-690f40-kserve-f8d77547f-rrldk\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") " pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.692796 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.692752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:43.842993 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:43.842883    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"]
Jul 06 16:08:43.845645 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:08:43.845611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ed76e5d_be64_49c0_adee_a971a27cc0a3.slice/crio-3cbebb3dcb8ce1bfbab3ab34897c4c1924f8e974b89b3fb0e6e9a6661b7833f0 WatchSource:0}: Error finding container 3cbebb3dcb8ce1bfbab3ab34897c4c1924f8e974b89b3fb0e6e9a6661b7833f0: Status 404 returned error can't find the container with id 3cbebb3dcb8ce1bfbab3ab34897c4c1924f8e974b89b3fb0e6e9a6661b7833f0
Jul 06 16:08:44.131181 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:44.131078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk" event={"ID":"1ed76e5d-be64-49c0-adee-a971a27cc0a3","Type":"ContainerStarted","Data":"c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7"}
Jul 06 16:08:44.131181 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:44.131128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk" event={"ID":"1ed76e5d-be64-49c0-adee-a971a27cc0a3","Type":"ContainerStarted","Data":"3cbebb3dcb8ce1bfbab3ab34897c4c1924f8e974b89b3fb0e6e9a6661b7833f0"}
Jul 06 16:08:44.150437 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:44.150378    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk" podStartSLOduration=1.150361882 podStartE2EDuration="1.150361882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:08:44.14830437 +0000 UTC m=+2515.961167149" watchObservedRunningTime="2026-07-06 16:08:44.150361882 +0000 UTC m=+2515.963224660"
Jul 06 16:08:53.692917 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:53.692871    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:53.693419 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:53.692940    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:53.705860 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:53.705829    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:08:54.198843 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:08:54.198811    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:09:40.085680 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:40.085640    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"]
Jul 06 16:09:40.975315 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:40.975273    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"]
Jul 06 16:09:55.106907 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.106859    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h" podUID="a2674f40-a3fa-4c41-aa7c-26e68b62173a" containerName="main" containerID="cri-o://1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74" gracePeriod=45
Jul 06 16:09:55.370545 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.370519    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:09:55.412409 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412374    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-model-cache\") pod \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") "
Jul 06 16:09:55.412598 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6h6p\" (UniqueName: \"kubernetes.io/projected/a2674f40-a3fa-4c41-aa7c-26e68b62173a-kube-api-access-q6h6p\") pod \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") "
Jul 06 16:09:55.412598 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412461    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tls-certs\") pod \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") "
Jul 06 16:09:55.412598 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412505    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tmp-dir\") pod \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") "
Jul 06 16:09:55.412598 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-dshm\") pod \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") "
Jul 06 16:09:55.412832 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-home\") pod \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\" (UID: \"a2674f40-a3fa-4c41-aa7c-26e68b62173a\") "
Jul 06 16:09:55.412832 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412714    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2674f40-a3fa-4c41-aa7c-26e68b62173a" (UID: "a2674f40-a3fa-4c41-aa7c-26e68b62173a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:55.412946 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.412927    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-home" (OuterVolumeSpecName: "home") pod "a2674f40-a3fa-4c41-aa7c-26e68b62173a" (UID: "a2674f40-a3fa-4c41-aa7c-26e68b62173a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:55.413029 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.413009    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:55.413082 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.413034    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:55.413123 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.413101    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2674f40-a3fa-4c41-aa7c-26e68b62173a" (UID: "a2674f40-a3fa-4c41-aa7c-26e68b62173a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:55.415290 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.415261    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2674f40-a3fa-4c41-aa7c-26e68b62173a" (UID: "a2674f40-a3fa-4c41-aa7c-26e68b62173a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:09:55.415421 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.415297    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2674f40-a3fa-4c41-aa7c-26e68b62173a-kube-api-access-q6h6p" (OuterVolumeSpecName: "kube-api-access-q6h6p") pod "a2674f40-a3fa-4c41-aa7c-26e68b62173a" (UID: "a2674f40-a3fa-4c41-aa7c-26e68b62173a"). InnerVolumeSpecName "kube-api-access-q6h6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:09:55.415421 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.415312    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-dshm" (OuterVolumeSpecName: "dshm") pod "a2674f40-a3fa-4c41-aa7c-26e68b62173a" (UID: "a2674f40-a3fa-4c41-aa7c-26e68b62173a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:55.514323 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.514283    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6h6p\" (UniqueName: \"kubernetes.io/projected/a2674f40-a3fa-4c41-aa7c-26e68b62173a-kube-api-access-q6h6p\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:55.514323 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.514316    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:55.514323 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.514330    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:55.514323 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.514337    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2674f40-a3fa-4c41-aa7c-26e68b62173a-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:55.559541 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.559501    2580 generic.go:356] "Generic (PLEG): container finished" podID="a2674f40-a3fa-4c41-aa7c-26e68b62173a" containerID="1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74" exitCode=0
Jul 06 16:09:55.559751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.559550    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"
Jul 06 16:09:55.559751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.559597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h" event={"ID":"a2674f40-a3fa-4c41-aa7c-26e68b62173a","Type":"ContainerDied","Data":"1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74"}
Jul 06 16:09:55.559751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.559649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h" event={"ID":"a2674f40-a3fa-4c41-aa7c-26e68b62173a","Type":"ContainerDied","Data":"8df80310a4067a15c628870ecd168dd0878277588244237cba9f9e30c4120abe"}
Jul 06 16:09:55.559751 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.559671    2580 scope.go:117] "RemoveContainer" containerID="1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74"
Jul 06 16:09:55.570173 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.570144    2580 scope.go:117] "RemoveContainer" containerID="1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74"
Jul 06 16:09:55.570493 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:09:55.570476    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74\": container with ID starting with 1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74 not found: ID does not exist" containerID="1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74"
Jul 06 16:09:55.570552 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.570505    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74"} err="failed to get container status \"1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74\": rpc error: code = NotFound desc = could not find container \"1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74\": container with ID starting with 1eccb0ab67b68a5cce01da850cdace5152f5d5dce3ab5dc6d506cd6688758a74 not found: ID does not exist"
Jul 06 16:09:55.584923 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.584882    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"]
Jul 06 16:09:55.590739 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.590687    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b75322/rate-test-model-b75322-kserve-76465c4b5f-xqf5h"]
Jul 06 16:09:55.995841 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:55.995797    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk" podUID="1ed76e5d-be64-49c0-adee-a971a27cc0a3" containerName="main" containerID="cri-o://c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7" gracePeriod=45
Jul 06 16:09:56.022059 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.022028    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2674f40-a3fa-4c41-aa7c-26e68b62173a" path="/var/lib/kubelet/pods/a2674f40-a3fa-4c41-aa7c-26e68b62173a/volumes"
Jul 06 16:09:56.256100 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.256023    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:09:56.321624 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94rk6\" (UniqueName: \"kubernetes.io/projected/1ed76e5d-be64-49c0-adee-a971a27cc0a3-kube-api-access-94rk6\") pod \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") "
Jul 06 16:09:56.321624 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-model-cache\") pod \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") "
Jul 06 16:09:56.321875 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321652    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-dshm\") pod \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") "
Jul 06 16:09:56.321875 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-home\") pod \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") "
Jul 06 16:09:56.321875 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321789    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tls-certs\") pod \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") "
Jul 06 16:09:56.321875 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tmp-dir\") pod \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\" (UID: \"1ed76e5d-be64-49c0-adee-a971a27cc0a3\") "
Jul 06 16:09:56.322087 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.321972    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ed76e5d-be64-49c0-adee-a971a27cc0a3" (UID: "1ed76e5d-be64-49c0-adee-a971a27cc0a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:56.322087 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.322041    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-home" (OuterVolumeSpecName: "home") pod "1ed76e5d-be64-49c0-adee-a971a27cc0a3" (UID: "1ed76e5d-be64-49c0-adee-a971a27cc0a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:56.322198 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.322151    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ed76e5d-be64-49c0-adee-a971a27cc0a3" (UID: "1ed76e5d-be64-49c0-adee-a971a27cc0a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:56.322198 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.322176    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-model-cache\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:56.322198 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.322195    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-home\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:56.324039 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.324012    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-dshm" (OuterVolumeSpecName: "dshm") pod "1ed76e5d-be64-49c0-adee-a971a27cc0a3" (UID: "1ed76e5d-be64-49c0-adee-a971a27cc0a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 16:09:56.324140 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.324031    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed76e5d-be64-49c0-adee-a971a27cc0a3-kube-api-access-94rk6" (OuterVolumeSpecName: "kube-api-access-94rk6") pod "1ed76e5d-be64-49c0-adee-a971a27cc0a3" (UID: "1ed76e5d-be64-49c0-adee-a971a27cc0a3"). InnerVolumeSpecName "kube-api-access-94rk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 16:09:56.324140 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.324057    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ed76e5d-be64-49c0-adee-a971a27cc0a3" (UID: "1ed76e5d-be64-49c0-adee-a971a27cc0a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 16:09:56.422788 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.422745    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94rk6\" (UniqueName: \"kubernetes.io/projected/1ed76e5d-be64-49c0-adee-a971a27cc0a3-kube-api-access-94rk6\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:56.422788 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.422784    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-dshm\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:56.422788 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.422795    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tls-certs\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:56.423030 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.422804    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ed76e5d-be64-49c0-adee-a971a27cc0a3-tmp-dir\") on node \"ip-10-0-141-57.ec2.internal\" DevicePath \"\""
Jul 06 16:09:56.566928 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.566833    2580 generic.go:356] "Generic (PLEG): container finished" podID="1ed76e5d-be64-49c0-adee-a971a27cc0a3" containerID="c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7" exitCode=0
Jul 06 16:09:56.566928 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.566882    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"
Jul 06 16:09:56.567161 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.566921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk" event={"ID":"1ed76e5d-be64-49c0-adee-a971a27cc0a3","Type":"ContainerDied","Data":"c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7"}
Jul 06 16:09:56.567161 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.566963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk" event={"ID":"1ed76e5d-be64-49c0-adee-a971a27cc0a3","Type":"ContainerDied","Data":"3cbebb3dcb8ce1bfbab3ab34897c4c1924f8e974b89b3fb0e6e9a6661b7833f0"}
Jul 06 16:09:56.567161 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.566982    2580 scope.go:117] "RemoveContainer" containerID="c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7"
Jul 06 16:09:56.577648 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.577625    2580 scope.go:117] "RemoveContainer" containerID="c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7"
Jul 06 16:09:56.577961 ip-10-0-141-57 kubenswrapper[2580]: E0706 16:09:56.577939    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7\": container with ID starting with c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7 not found: ID does not exist" containerID="c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7"
Jul 06 16:09:56.578006 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.577972    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7"} err="failed to get container status \"c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7\": rpc error: code = NotFound desc = could not find container \"c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7\": container with ID starting with c422f2918d007cce05d3a099372dfa5519dba6f7345abb5989ef3843d097e3a7 not found: ID does not exist"
Jul 06 16:09:56.593257 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.593226    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"]
Jul 06 16:09:56.596618 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:56.596584    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-690f40/rate-test-model-690f40-kserve-f8d77547f-rrldk"]
Jul 06 16:09:58.021550 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:09:58.021515    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ed76e5d-be64-49c0-adee-a971a27cc0a3" path="/var/lib/kubelet/pods/1ed76e5d-be64-49c0-adee-a971a27cc0a3/volumes"
Jul 06 16:10:37.440011 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:37.439978    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54c8db6ff8-q2cw6_8d7bf7b4-3b68-44f3-bb19-9940880d1966/authorino/0.log"
Jul 06 16:10:43.303590 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:43.303563    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8frj8_8788900f-ff02-4349-96d0-b22184666734/server/0.log"
Jul 06 16:10:43.436658 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:43.436629    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tqwl7_1a3665d2-dfb6-4704-847a-4b9ee5a376a8/manager/0.log"
Jul 06 16:10:43.808224 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:43.808189    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bdc85857f-c7nbn_6bacaebb-2ceb-4b70-a61d-2708192816a6/copy-manifests/0.log"
Jul 06 16:10:43.830130 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:43.830099    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bdc85857f-c7nbn_6bacaebb-2ceb-4b70-a61d-2708192816a6/manager/0.log"
Jul 06 16:10:44.988969 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:44.988941    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s_08310f53-b9cd-4347-8270-c10f252ae27a/util/0.log"
Jul 06 16:10:45.002801 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.002750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s_08310f53-b9cd-4347-8270-c10f252ae27a/pull/0.log"
Jul 06 16:10:45.019006 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.018927    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s_08310f53-b9cd-4347-8270-c10f252ae27a/extract/0.log"
Jul 06 16:10:45.146305 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.146278    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77_e9e28b80-09ab-403c-8197-c6a95157993c/util/0.log"
Jul 06 16:10:45.157529 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.157489    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77_e9e28b80-09ab-403c-8197-c6a95157993c/pull/0.log"
Jul 06 16:10:45.171330 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.171303    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77_e9e28b80-09ab-403c-8197-c6a95157993c/extract/0.log"
Jul 06 16:10:45.295505 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.295425    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99_a3df8aa2-76eb-4fdd-8d60-14f47bb68725/util/0.log"
Jul 06 16:10:45.324711 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.324681    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99_a3df8aa2-76eb-4fdd-8d60-14f47bb68725/pull/0.log"
Jul 06 16:10:45.335807 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.335780    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99_a3df8aa2-76eb-4fdd-8d60-14f47bb68725/extract/0.log"
Jul 06 16:10:45.459549 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.459520    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw_52e0dde8-8d93-4803-8578-0dbf53035aef/util/0.log"
Jul 06 16:10:45.474016 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.473991    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw_52e0dde8-8d93-4803-8578-0dbf53035aef/pull/0.log"
Jul 06 16:10:45.488426 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.488403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw_52e0dde8-8d93-4803-8578-0dbf53035aef/extract/0.log"
Jul 06 16:10:45.610308 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.610224    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54c8db6ff8-q2cw6_8d7bf7b4-3b68-44f3-bb19-9940880d1966/authorino/0.log"
Jul 06 16:10:45.983288 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:45.983262    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qlb8k_3e2049b8-4c34-4c1e-a206-205fa2188401/kuadrant-console-plugin/0.log"
Jul 06 16:10:47.091635 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:47.091607    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-864987f78f-gd98h_b6e32545-9622-448d-b622-15563ef383d4/kube-auth-proxy/0.log"
Jul 06 16:10:47.459088 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:47.459050    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t6jb5_bfa257ca-1566-4621-97ea-53e85de26e19/payload-pre-processing/0.log"
Jul 06 16:10:47.696533 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:47.696494    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7468768f6c-cbztv_42d7fee8-aac1-4d87-9034-8632800488f7/router/0.log"
Jul 06 16:10:48.043360 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:48.043331    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lh9bl_5b08b09c-9997-42e4-a431-ad4e838e9cee/main/0.log"
Jul 06 16:10:48.276731 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:48.276686    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-kvht5_902b2e7c-5c0b-4d1a-a6fd-566c44004a98/main/0.log"
Jul 06 16:10:48.633369 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:48.633335    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-62r5n_d9bac989-594a-4b62-b074-e474309f294e/main/0.log"
Jul 06 16:10:53.168846 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.168808    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-96f6w/must-gather-8m64b"]
Jul 06 16:10:53.169249 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.169199    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2674f40-a3fa-4c41-aa7c-26e68b62173a" containerName="main"
Jul 06 16:10:53.169249 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.169213    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2674f40-a3fa-4c41-aa7c-26e68b62173a" containerName="main"
Jul 06 16:10:53.169249 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.169240    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ed76e5d-be64-49c0-adee-a971a27cc0a3" containerName="main"
Jul 06 16:10:53.169249 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.169246    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ed76e5d-be64-49c0-adee-a971a27cc0a3" containerName="main"
Jul 06 16:10:53.169387 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.169309    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ed76e5d-be64-49c0-adee-a971a27cc0a3" containerName="main"
Jul 06 16:10:53.169387 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.169320    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2674f40-a3fa-4c41-aa7c-26e68b62173a" containerName="main"
Jul 06 16:10:53.174007 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.173986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.180402 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.180373    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-96f6w\"/\"default-dockercfg-msmgz\""
Jul 06 16:10:53.186771 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.186695    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-96f6w\"/\"kube-root-ca.crt\""
Jul 06 16:10:53.196040 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.196009    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-96f6w/must-gather-8m64b"]
Jul 06 16:10:53.206884 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.206848    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-96f6w\"/\"openshift-service-ca.crt\""
Jul 06 16:10:53.269764 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.269693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldqzc\" (UniqueName: \"kubernetes.io/projected/b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139-kube-api-access-ldqzc\") pod \"must-gather-8m64b\" (UID: \"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139\") " pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.269942 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.269775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139-must-gather-output\") pod \"must-gather-8m64b\" (UID: \"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139\") " pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.370525 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.370464    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldqzc\" (UniqueName: \"kubernetes.io/projected/b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139-kube-api-access-ldqzc\") pod \"must-gather-8m64b\" (UID: \"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139\") " pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.370525 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.370526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139-must-gather-output\") pod \"must-gather-8m64b\" (UID: \"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139\") " pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.370974 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.370951    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139-must-gather-output\") pod \"must-gather-8m64b\" (UID: \"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139\") " pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.385619 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.385584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldqzc\" (UniqueName: \"kubernetes.io/projected/b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139-kube-api-access-ldqzc\") pod \"must-gather-8m64b\" (UID: \"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139\") " pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.483898 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.483800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-96f6w/must-gather-8m64b"
Jul 06 16:10:53.638581 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.638474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-96f6w/must-gather-8m64b"]
Jul 06 16:10:53.641230 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:10:53.641191    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3d5c0c0_0ce4_49d6_9e08_6dd24fe87139.slice/crio-7a0446ed2c8ef029a90cb71605195353e0067265455c0a1cf2ea83656fe28199 WatchSource:0}: Error finding container 7a0446ed2c8ef029a90cb71605195353e0067265455c0a1cf2ea83656fe28199: Status 404 returned error can't find the container with id 7a0446ed2c8ef029a90cb71605195353e0067265455c0a1cf2ea83656fe28199
Jul 06 16:10:53.643145 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.643127    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 16:10:53.906609 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:53.906573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-96f6w/must-gather-8m64b" event={"ID":"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139","Type":"ContainerStarted","Data":"7a0446ed2c8ef029a90cb71605195353e0067265455c0a1cf2ea83656fe28199"}
Jul 06 16:10:55.924937 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:55.924888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-96f6w/must-gather-8m64b" event={"ID":"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139","Type":"ContainerStarted","Data":"8ec30abefdc39ab43ff9d4c2d55c3db85b0e5eb1800fa7079a7da67b142d50c7"}
Jul 06 16:10:55.925437 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:55.924970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-96f6w/must-gather-8m64b" event={"ID":"b3d5c0c0-0ce4-49d6-9e08-6dd24fe87139","Type":"ContainerStarted","Data":"106e63adc6598d7fa5b2b6ec6b9136ab4824abab0f1709ef391fa0489e5dba86"}
Jul 06 16:10:55.955085 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:55.955030    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-96f6w/must-gather-8m64b" podStartSLOduration=1.623405247 podStartE2EDuration="2.955014927s" totalImagesPullingTime="1.33160968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:10:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:10:55.954559363 +0000 UTC m=+2647.767422141" watchObservedRunningTime="2026-07-06 16:10:55.955014927 +0000 UTC m=+2647.767877730"
Jul 06 16:10:57.010614 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:57.010579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5d7zj_453f039d-a9cd-468d-88d5-34edaa9d8ef0/global-pull-secret-syncer/0.log"
Jul 06 16:10:57.387675 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:57.387642    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xz86f_fbb51933-4cc0-4171-a916-676625b030ef/konnectivity-agent/0.log"
Jul 06 16:10:57.490319 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:10:57.490279    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-57.ec2.internal_77b865350d5ba6c456bb119a49e5b239/haproxy/0.log"
Jul 06 16:11:01.829195 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:01.829157    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s_08310f53-b9cd-4347-8270-c10f252ae27a/extract/0.log"
Jul 06 16:11:01.886047 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:01.886015    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s_08310f53-b9cd-4347-8270-c10f252ae27a/util/0.log"
Jul 06 16:11:01.920894 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:01.920866    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596bt8s_08310f53-b9cd-4347-8270-c10f252ae27a/pull/0.log"
Jul 06 16:11:01.995682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:01.995655    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77_e9e28b80-09ab-403c-8197-c6a95157993c/extract/0.log"
Jul 06 16:11:02.027477 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.027424    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77_e9e28b80-09ab-403c-8197-c6a95157993c/util/0.log"
Jul 06 16:11:02.075407 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.075306    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qcd77_e9e28b80-09ab-403c-8197-c6a95157993c/pull/0.log"
Jul 06 16:11:02.128436 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.128407    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99_a3df8aa2-76eb-4fdd-8d60-14f47bb68725/extract/0.log"
Jul 06 16:11:02.167166 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.167138    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99_a3df8aa2-76eb-4fdd-8d60-14f47bb68725/util/0.log"
Jul 06 16:11:02.223198 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.223168    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z7v99_a3df8aa2-76eb-4fdd-8d60-14f47bb68725/pull/0.log"
Jul 06 16:11:02.274256 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.274199    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw_52e0dde8-8d93-4803-8578-0dbf53035aef/extract/0.log"
Jul 06 16:11:02.311075 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.311044    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw_52e0dde8-8d93-4803-8578-0dbf53035aef/util/0.log"
Jul 06 16:11:02.347183 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.347095    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gwwtw_52e0dde8-8d93-4803-8578-0dbf53035aef/pull/0.log"
Jul 06 16:11:02.723944 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.723882    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54c8db6ff8-q2cw6_8d7bf7b4-3b68-44f3-bb19-9940880d1966/authorino/0.log"
Jul 06 16:11:02.830001 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:02.829968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qlb8k_3e2049b8-4c34-4c1e-a206-205fa2188401/kuadrant-console-plugin/0.log"
Jul 06 16:11:04.849602 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:04.849556    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/alertmanager/0.log"
Jul 06 16:11:04.888400 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:04.888299    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/config-reloader/0.log"
Jul 06 16:11:04.925173 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:04.925134    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/kube-rbac-proxy-web/0.log"
Jul 06 16:11:04.964448 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:04.964417    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/kube-rbac-proxy/0.log"
Jul 06 16:11:05.002115 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.002069    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/kube-rbac-proxy-metric/0.log"
Jul 06 16:11:05.033413 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.033386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/prom-label-proxy/0.log"
Jul 06 16:11:05.073712 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.073668    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5e56ed-33df-494e-aa43-d9aeaa0bc1ff/init-config-reloader/0.log"
Jul 06 16:11:05.132848 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.132808    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-x8d7q_6c04b5fb-1ad5-4bf7-95f3-e56e9eae47c9/cluster-monitoring-operator/0.log"
Jul 06 16:11:05.167628 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.167593    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-v6r42_8ed5bba2-e1c8-4c56-86a8-1de04638eafa/kube-state-metrics/0.log"
Jul 06 16:11:05.196444 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.196411    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-v6r42_8ed5bba2-e1c8-4c56-86a8-1de04638eafa/kube-rbac-proxy-main/0.log"
Jul 06 16:11:05.225314 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.225277    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-v6r42_8ed5bba2-e1c8-4c56-86a8-1de04638eafa/kube-rbac-proxy-self/0.log"
Jul 06 16:11:05.281135 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.281088    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75cd77d48f-cv6t8_0397f69d-33f9-4bed-be4e-4cbe1497574f/metrics-server/0.log"
Jul 06 16:11:05.540926 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.540834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jtp4m_0902663f-3122-4f56-98a7-770cf809d201/node-exporter/0.log"
Jul 06 16:11:05.581674 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.581642    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jtp4m_0902663f-3122-4f56-98a7-770cf809d201/kube-rbac-proxy/0.log"
Jul 06 16:11:05.623142 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.623110    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jtp4m_0902663f-3122-4f56-98a7-770cf809d201/init-textfile/0.log"
Jul 06 16:11:05.831015 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.830930    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-t2gfl_fff5e63b-e081-4365-9e55-d7a431d73d23/kube-rbac-proxy-main/0.log"
Jul 06 16:11:05.880373 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.880337    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-t2gfl_fff5e63b-e081-4365-9e55-d7a431d73d23/kube-rbac-proxy-self/0.log"
Jul 06 16:11:05.917310 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:05.917280    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-t2gfl_fff5e63b-e081-4365-9e55-d7a431d73d23/openshift-state-metrics/0.log"
Jul 06 16:11:06.384576 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.384545    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-kvp6q_49d68ccf-ff59-4f36-9014-d1cb964ac8fb/prometheus-operator-admission-webhook/0.log"
Jul 06 16:11:06.528823 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.528787    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f74cb6d84-vbh6m_c2000c95-00d3-4b24-a9f6-d47fea3a3030/thanos-query/0.log"
Jul 06 16:11:06.566055 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.566026    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f74cb6d84-vbh6m_c2000c95-00d3-4b24-a9f6-d47fea3a3030/kube-rbac-proxy-web/0.log"
Jul 06 16:11:06.600957 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.600925    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f74cb6d84-vbh6m_c2000c95-00d3-4b24-a9f6-d47fea3a3030/kube-rbac-proxy/0.log"
Jul 06 16:11:06.644380 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.644289    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f74cb6d84-vbh6m_c2000c95-00d3-4b24-a9f6-d47fea3a3030/prom-label-proxy/0.log"
Jul 06 16:11:06.672189 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.672162    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f74cb6d84-vbh6m_c2000c95-00d3-4b24-a9f6-d47fea3a3030/kube-rbac-proxy-rules/0.log"
Jul 06 16:11:06.697971 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:06.697938    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f74cb6d84-vbh6m_c2000c95-00d3-4b24-a9f6-d47fea3a3030/kube-rbac-proxy-metrics/0.log"
Jul 06 16:11:07.933402 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:07.933376    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-hdj6t_baaca221-afba-4397-a744-e5ef87e1f5df/networking-console-plugin/0.log"
Jul 06 16:11:08.029942 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.029903    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"]
Jul 06 16:11:08.036275 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.036233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.042457 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.042421    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"]
Jul 06 16:11:08.139145 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.139107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-podres\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.139145 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.139155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-proc\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.139419 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.139249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-lib-modules\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.139419 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.139312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jwn7\" (UniqueName: \"kubernetes.io/projected/09cfee34-7aa1-416a-a502-f2a27abd972c-kube-api-access-7jwn7\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.139419 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.139349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-sys\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240406 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-podres\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240406 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-proc\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240406 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-lib-modules\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jwn7\" (UniqueName: \"kubernetes.io/projected/09cfee34-7aa1-416a-a502-f2a27abd972c-kube-api-access-7jwn7\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-sys\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-podres\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240572    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-lib-modules\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240575    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-sys\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.240682 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.240615    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/09cfee34-7aa1-416a-a502-f2a27abd972c-proc\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.254677 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.254645    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jwn7\" (UniqueName: \"kubernetes.io/projected/09cfee34-7aa1-416a-a502-f2a27abd972c-kube-api-access-7jwn7\") pod \"perf-node-gather-daemonset-rkkmx\" (UID: \"09cfee34-7aa1-416a-a502-f2a27abd972c\") " pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.356223 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.355943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:08.546316 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:08.546188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"]
Jul 06 16:11:08.550919 ip-10-0-141-57 kubenswrapper[2580]: W0706 16:11:08.550880    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod09cfee34_7aa1_416a_a502_f2a27abd972c.slice/crio-00197160f333efe509097c3384b570cb2b72c23bdc8fa6d89d46912fce1231de WatchSource:0}: Error finding container 00197160f333efe509097c3384b570cb2b72c23bdc8fa6d89d46912fce1231de: Status 404 returned error can't find the container with id 00197160f333efe509097c3384b570cb2b72c23bdc8fa6d89d46912fce1231de
Jul 06 16:11:09.039084 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:09.039048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx" event={"ID":"09cfee34-7aa1-416a-a502-f2a27abd972c","Type":"ContainerStarted","Data":"ed7e6bae88db299af4726fdd8d7c570c7117ec2af44a749ce60589114467eab8"}
Jul 06 16:11:09.039510 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:09.039102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx" event={"ID":"09cfee34-7aa1-416a-a502-f2a27abd972c","Type":"ContainerStarted","Data":"00197160f333efe509097c3384b570cb2b72c23bdc8fa6d89d46912fce1231de"}
Jul 06 16:11:09.039510 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:09.039209    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:09.067476 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:09.067419    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx" podStartSLOduration=2.067400188 podStartE2EDuration="2.067400188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:11:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:11:09.064875866 +0000 UTC m=+2660.877738645" watchObservedRunningTime="2026-07-06 16:11:09.067400188 +0000 UTC m=+2660.880262965"
Jul 06 16:11:09.943980 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:09.943942    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-dd24v_c0d4dba4-08b8-45c2-986d-15c6bd46954f/volume-data-source-validator/0.log"
Jul 06 16:11:11.369192 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:11.369148    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8n47_328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e/dns/0.log"
Jul 06 16:11:11.414458 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:11.414415    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8n47_328c2ae1-5a8d-49b2-84b0-7ff8611a9e1e/kube-rbac-proxy/0.log"
Jul 06 16:11:11.596811 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:11.596780    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lf2hj_4af80c0b-5938-4740-8c22-dfac3b02e1d0/dns-node-resolver/0.log"
Jul 06 16:11:12.401491 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:12.401455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fkmm6_0c2c1f01-a050-4040-a6c8-b39fd086a324/node-ca/0.log"
Jul 06 16:11:13.776121 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:13.776094    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-864987f78f-gd98h_b6e32545-9622-448d-b622-15563ef383d4/kube-auth-proxy/0.log"
Jul 06 16:11:13.914303 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:13.914267    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t6jb5_bfa257ca-1566-4621-97ea-53e85de26e19/payload-pre-processing/0.log"
Jul 06 16:11:13.999070 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:13.999041    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7468768f6c-cbztv_42d7fee8-aac1-4d87-9034-8632800488f7/router/0.log"
Jul 06 16:11:14.850036 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:14.849994    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5t8j5_7b625305-ec07-489b-9d17-6b0a5262a856/serve-healthcheck-canary/0.log"
Jul 06 16:11:15.059299 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:15.059269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-96f6w/perf-node-gather-daemonset-rkkmx"
Jul 06 16:11:15.583642 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:15.583612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g5ccn_58ed0bdd-c406-4dbe-8659-5595955f68ff/kube-rbac-proxy/0.log"
Jul 06 16:11:15.618301 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:15.618269    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g5ccn_58ed0bdd-c406-4dbe-8659-5595955f68ff/exporter/0.log"
Jul 06 16:11:15.645541 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:15.645503    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g5ccn_58ed0bdd-c406-4dbe-8659-5595955f68ff/extractor/0.log"
Jul 06 16:11:18.671322 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.671292    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8frj8_8788900f-ff02-4349-96d0-b22184666734/server/0.log"
Jul 06 16:11:18.763035 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.762934    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tqwl7_1a3665d2-dfb6-4704-847a-4b9ee5a376a8/manager/0.log"
Jul 06 16:11:18.790384 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.790358    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-864987f78f-gd98h_b6e32545-9622-448d-b622-15563ef383d4/kube-auth-proxy/0.log"
Jul 06 16:11:18.885717 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.885672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t6jb5_bfa257ca-1566-4621-97ea-53e85de26e19/payload-pre-processing/0.log"
Jul 06 16:11:18.906280 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.906227    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bdc85857f-c7nbn_6bacaebb-2ceb-4b70-a61d-2708192816a6/manager/0.log"
Jul 06 16:11:18.932939 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.932902    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bdc85857f-c7nbn_6bacaebb-2ceb-4b70-a61d-2708192816a6/copy-manifests/0.log"
Jul 06 16:11:18.940033 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:18.940002    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7468768f6c-cbztv_42d7fee8-aac1-4d87-9034-8632800488f7/router/0.log"
Jul 06 16:11:20.403686 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:20.403656    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-864987f78f-gd98h_b6e32545-9622-448d-b622-15563ef383d4/kube-auth-proxy/0.log"
Jul 06 16:11:20.503968 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:20.503935    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-t6jb5_bfa257ca-1566-4621-97ea-53e85de26e19/payload-pre-processing/0.log"
Jul 06 16:11:20.558235 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:20.558156    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7468768f6c-cbztv_42d7fee8-aac1-4d87-9034-8632800488f7/router/0.log"
Jul 06 16:11:20.598510 ip-10-0-141-57 kubenswrapper[2580]: I0706 16:11:20.598482    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-df4586b48-zngsx_0729f95a-995d-4c7f-99e1-fe51e18dcdbe/manager/0.log"