Jul 29 10:28:06.880693 ip-10-0-143-36 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 10:28:07.331666 ip-10-0-143-36 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 10:28:07.331666 ip-10-0-143-36 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 10:28:07.331666 ip-10-0-143-36 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 10:28:07.331666 ip-10-0-143-36 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 10:28:07.331666 ip-10-0-143-36 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 10:28:07.332829 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.332746    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 10:28:07.337888 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337865    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:07.337888 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337887    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:07.337888 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337891    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337895    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337898    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337901    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337904    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337907    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337910    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337913    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337915    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337919    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337922    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337925    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337927    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337930    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337933    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337936    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337939    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337942    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337946    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:07.337995 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337950    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337952    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337955    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337958    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337961    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337963    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337966    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337969    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337971    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337974    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337976    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337980    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337982    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337986    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337989    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337993    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337996    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.337999    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338005    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338009    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:07.338468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338012    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338016    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338019    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338022    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338026    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338030    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338033    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338036    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338039    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338042    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338045    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338048    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338050    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338055    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338058    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338060    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338063    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338066    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338069    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:07.338962 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338071    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338074    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338077    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338080    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338082    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338085    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338088    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338091    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338094    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338098    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338102    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338106    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338109    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338112    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338115    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338118    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338121    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338124    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338127    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338130    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:07.339436 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338132    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338135    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338138    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338140    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338143    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338145    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338148    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338151    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338153    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338156    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338158    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338161    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338164    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338167    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338169    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338172    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338175    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338178    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338180    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338930    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:07.339923 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338937    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338940    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338943    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338946    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338954    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338957    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338960    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338963    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338966    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338969    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338972    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338975    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338977    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338981    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338985    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338988    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338990    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338993    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.338998    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:07.340430 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339001    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339004    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339008    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339011    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339014    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339016    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339019    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339022    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339025    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339027    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339030    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339033    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339035    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339038    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339041    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339044    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339046    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339049    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339057    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339060    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:07.340960 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339063    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339065    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339068    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339070    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339074    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339078    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339081    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339083    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339086    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339089    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339092    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339094    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339097    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339101    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339104    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339106    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339109    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339111    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339114    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:07.341468 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339117    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339119    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339122    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339125    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339127    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339130    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339133    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339136    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339138    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339141    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339143    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339146    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339150    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339152    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339156    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339158    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339160    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339163    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339166    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339168    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:07.341931 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339171    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339173    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339176    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339181    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339184    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339186    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339189    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339191    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339194    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339198    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339200    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339203    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339205    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339208    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339211    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339214    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339216    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339219    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339222    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339224    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:07.342422 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339696    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339704    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339708    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339711    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339715    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339718    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339721    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339724    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339727    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339730    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339733    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339735    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339738    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339741    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339744    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339749    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339752    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339755    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339757    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339760    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:07.342927 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339763    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339766    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339768    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339771    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339773    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339776    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339779    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339781    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339786    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339790    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339793    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339796    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339798    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339801    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339804    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339807    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339810    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339822    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339825    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:07.343419 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339828    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339830    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339833    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339835    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339838    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339841    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339843    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339847    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339852    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339855    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339866    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339869    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339872    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339874    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339877    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339880    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339882    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339885    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339888    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:07.343887 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339891    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339894    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339897    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339901    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339904    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339907    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339909    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339912    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339914    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339917    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339920    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339922    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339926    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339929    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339932    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339934    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339937    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339940    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339948    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339950    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:07.344355 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339953    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339956    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339959    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339961    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339964    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339967    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339969    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339972    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339974    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339977    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339979    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339982    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339984    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339987    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339991    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339994    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.339997    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340000    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340003    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340006    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:07.344862 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340008    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340096    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340107    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340117    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340124    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340131    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340135    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340139    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340144    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340147    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340150    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340154    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340158    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340161    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340164    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340167    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340171    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340174    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340177    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340181    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340184    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340187    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340190    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340194    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 10:28:07.345349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340198    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340201    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340204    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340208    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340211    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340214    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340217    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340221    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340224    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340228    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340231    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340234    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340238    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340241    2562 flags.go:64] FLAG: --enable-server="true"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340244    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340249    2562 flags.go:64] FLAG: --event-burst="100"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340253    2562 flags.go:64] FLAG: --event-qps="50"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340255    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340259    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340262    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340266    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340269    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340272    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340275    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340278    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 10:28:07.345935 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340281    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340284    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340288    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340291    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340294    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340297    2562 flags.go:64] FLAG: --feature-gates=""
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340301    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340304    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340307    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340313    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340316    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340331    2562 flags.go:64] FLAG: --help="false"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340334    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340338    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340341    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340344    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340347    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340351    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340354    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340357    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340360    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340363    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340366    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340370    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 10:28:07.346595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340373    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340375    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340379    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340382    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340385    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340388    2562 flags.go:64] FLAG: --lock-file=""
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340391    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340394    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340397    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340402    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340405    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340408    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340411    2562 flags.go:64] FLAG: --logging-format="text"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340413    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340417    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340420    2562 flags.go:64] FLAG: --manifest-url=""
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340423    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340429    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340432    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340436    2562 flags.go:64] FLAG: --max-pods="110"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340439    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340442    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340445    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340449    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340452    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 10:28:07.347203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340455    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340458    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340466    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340469    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340473    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340476    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340480    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340486    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340490    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340493    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340496    2562 flags.go:64] FLAG: --port="10250"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340499    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340502    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-072cc39c352a4bb87"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340506    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340508    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340511    2562 flags.go:64] FLAG: --register-node="true"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340514    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340518    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340521    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340524    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340527    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340531    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340534    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340537    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340542    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 10:28:07.347849 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340545    2562 flags.go:64] FLAG: --runonce="false"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340548    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340551    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340554    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340557    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340560    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340563    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340566    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340570    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340573    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340576    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340579    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340584    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340587    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340591    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340594    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340599    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340602    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340620    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340626    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340629    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340632    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340635    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340638    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340641    2562 flags.go:64] FLAG: --v="2"
Jul 29 10:28:07.348486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340645    2562 flags.go:64] FLAG: --version="false"
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340649    2562 flags.go:64] FLAG: --vmodule=""
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340653    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.340656    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340777    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340781    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340785    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340789    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340792    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340795    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340798    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340800    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340803    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340806    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340809    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340812    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340816    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340820    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340823    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340828    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:07.349092 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340831    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340834    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340838    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340840    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340843    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340846    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340848    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340851    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340854    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340857    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340859    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340862    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340864    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340867    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340870    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340872    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340875    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340877    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340882    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:07.349591 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340886    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340889    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340892    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340895    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340898    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340901    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340904    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340907    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340909    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340912    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340915    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340918    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340923    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340925    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340928    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340931    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340934    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340936    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340939    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340942    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:07.350083 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340944    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340947    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340949    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340952    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340955    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340957    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340960    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340963    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340965    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340968    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340971    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340976    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340979    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340981    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340984    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340986    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340989    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340991    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340994    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340996    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:07.350596 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.340999    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341002    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341004    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341007    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341011    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341014    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341016    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341019    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341022    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341025    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341028    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341030    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341033    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341035    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341038    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341041    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341043    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341046    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341048    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341051    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:07.351090 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341053    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341056    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341059    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.341065    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.341879    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.349166    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.349189    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.350099    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.350108    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350174    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350180    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350184    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350187    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350190    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:07.351599 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350193    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350196    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350198    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350201    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350204    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350207    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350210    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350212    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350215    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350217    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350220    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350222    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350225    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350228    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350230    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350233    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350235    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350238    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350240    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350243    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:07.351948 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350245    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350248    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350250    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350253    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350258    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350260    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350265    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350269    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350272    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350275    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350278    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350280    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350283    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350286    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350288    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350291    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350293    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350296    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350298    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:07.352449 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350301    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350304    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350306    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350309    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350312    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350314    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350329    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350333    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350336    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350338    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350341    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350345    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350349    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350353    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350356    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350359    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350362    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350365    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350369    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:07.352915 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350371    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350374    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350377    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350379    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350382    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350385    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350388    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350391    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350393    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350396    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350399    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350401    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350404    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350407    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350409    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350412    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350414    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350417    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350419    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350422    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:07.353427 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350424    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350427    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350430    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350432    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350435    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350438    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350441    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350444    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350448    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350451    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350455    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350458    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350461    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350464    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350467    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:07.353949 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350469    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.350475    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350632    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350639    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350643    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350646    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350649    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350652    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350655    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350657    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350660    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350663    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350666    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350668    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350671    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 10:28:07.354353 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350674    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350677    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350679    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350683    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350687    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350690    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350692    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350695    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350698    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350707    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350711    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350713    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350716    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350719    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350721    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350725    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350728    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350731    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350734    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 10:28:07.354798 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350736    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350739    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350742    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350744    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350747    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350749    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350752    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350755    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350757    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350759    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350762    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350765    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350767    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350770    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350773    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350776    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350778    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350781    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350783    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 10:28:07.355537 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350787    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350791    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350794    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350796    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350799    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350802    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350805    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350808    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350811    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350814    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350817    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350820    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350823    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350825    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350828    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350830    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350833    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350836    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350838    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 10:28:07.356231 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350841    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350843    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350846    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350848    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350851    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350853    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350856    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350859    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350861    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350864    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350866    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350869    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350872    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350874    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350877    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350880    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350882    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350885    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350888    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350891    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 10:28:07.356901 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350893    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350896    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350898    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350901    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350904    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350906    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350909    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350912    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:07.350915    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.350919    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.351063    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.353628    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.356158    2562 server.go:1019] "Starting client certificate rotation"
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.356274    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 10:28:07.357402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.356362    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 10:28:07.360254 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.360230    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 10:28:07.364449 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.364424    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 10:28:07.378768 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.378739    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 29 10:28:07.382264 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.382247    2562 log.go:25] "Validated CRI v1 image API"
Jul 29 10:28:07.383152 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.383134    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 10:28:07.386097 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.386061    2562 fs.go:135] Filesystem UUIDs: map[3bdf857c-eb91-4fb1-a746-126d1ecfe6ad:/dev/nvme0n1p3 3cacbfdd-856f-4a8c-acb4-68a6f4a88b4c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 29 10:28:07.386195 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.386091    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 10:28:07.386908 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.386880    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 10:28:07.392515 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.392381    2562 manager.go:216] Machine: {Timestamp:2026-07-29 10:28:07.390902034 +0000 UTC m=+0.379751480 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3078840 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2706d6c0451ef8ea4e905d31b61f2f SystemUUID:ec2706d6-c045-1ef8-ea4e-905d31b61f2f BootID:1d6f0200-42fc-4aa9-bbca-2964be4d0952 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:ba:4a:7a:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:ba:4a:7a:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:4c:18:c9:c6:be Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 10:28:07.392515 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.392502    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 10:28:07.392665 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.392608    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 10:28:07.396207 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.396165    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 10:28:07.396375 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.396208    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-36.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 10:28:07.397067 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.397054    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 10:28:07.397103 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.397068    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 10:28:07.397103 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.397078    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 10:28:07.397103 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.397091    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 10:28:07.399004 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.398989    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 10:28:07.403762 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.403744    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 10:28:07.403908 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.403897    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 10:28:07.404571 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.404561    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 10:28:07.404619 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.404575    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 10:28:07.404619 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.404602    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 10:28:07.404619 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.404612    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 29 10:28:07.404712 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.404629    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 10:28:07.405060 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.405046    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 10:28:07.405096 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.405072    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 10:28:07.408718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.408688    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 10:28:07.410695 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.410679    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 10:28:07.410748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.410719    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 10:28:07.411568 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411557    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 10:28:07.411602 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411573    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 10:28:07.411602 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411579    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 10:28:07.411602 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411585    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 10:28:07.411602 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411594    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 10:28:07.411602 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411600    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 10:28:07.411727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411612    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 10:28:07.411727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411620    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 10:28:07.411727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411627    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 10:28:07.411727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411632    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 10:28:07.411727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411641    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 10:28:07.411727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.411645    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 10:28:07.412473 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.412462    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 10:28:07.412515 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.412473    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 10:28:07.415964 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.415948    2562 server.go:1268] "Started kubelet"
Jul 29 10:28:07.416101 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.416062    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 10:28:07.416171 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.416116    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 10:28:07.416211 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.416199    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 10:28:07.416927 ip-10-0-143-36 systemd[1]: Started Kubernetes Kubelet.
Jul 29 10:28:07.417972 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.417953    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 10:28:07.418173 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.418159    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 10:28:07.418685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.418664    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-36.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 10:28:07.418797 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.418719    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 10:28:07.418797 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.418751    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 10:28:07.423170 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.423147    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 10:28:07.423743 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.422601    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-36.ec2.internal.18c6bbe35882c983  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-36.ec2.internal,UID:ip-10-0-143-36.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-36.ec2.internal,},FirstTimestamp:2026-07-29 10:28:07.415916931 +0000 UTC m=+0.404766377,LastTimestamp:2026-07-29 10:28:07.415916931 +0000 UTC m=+0.404766377,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-36.ec2.internal,}"
Jul 29 10:28:07.427880 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.427860    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 10:28:07.427880 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.427872    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 10:28:07.428036 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.428015    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.428736 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.428720    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 10:28:07.428826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.428723    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 10:28:07.428826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.428763    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 10:28:07.428826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.428795    2562 factory.go:55] Registering systemd factory
Jul 29 10:28:07.428826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.428814    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 29 10:28:07.429053 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429040    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 10:28:07.429053 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429052    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 10:28:07.429133 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429090    2562 factory.go:153] Registering CRI-O factory
Jul 29 10:28:07.429133 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429105    2562 factory.go:223] Registration of the crio container factory successfully
Jul 29 10:28:07.429188 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429159    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 10:28:07.429188 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429183    2562 factory.go:103] Registering Raw factory
Jul 29 10:28:07.429240 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429198    2562 manager.go:1196] Started watching for new ooms in manager
Jul 29 10:28:07.429816 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.429801    2562 manager.go:349] Starting recovery of all containers
Jul 29 10:28:07.430399 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.430378    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 10:28:07.430877 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.430855    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 10:28:07.436938 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.436902    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5vfjx"
Jul 29 10:28:07.442417 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.442395    2562 manager.go:354] Recovery completed
Jul 29 10:28:07.445502 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.445480    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5vfjx"
Jul 29 10:28:07.447235 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.447221    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.449238 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449089    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.449353 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449259    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.449353 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449276    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.449947 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449934    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 10:28:07.449947 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449945    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 10:28:07.450047 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449963    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 10:28:07.450047 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.449964    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.450770 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.450757    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.450820 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.450781    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.450820 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.450791    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.451912 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.451900    2562 policy_none.go:49] "None policy: Start"
Jul 29 10:28:07.451948 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.451917    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 10:28:07.451948 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.451928    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 10:28:07.453974 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.453963    2562 policy_none.go:47] "Start"
Jul 29 10:28:07.489210 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489188    2562 manager.go:337] "Starting Device Plugin manager"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.489240    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489255    2562 server.go:87] "Starting device plugin registration server"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489580    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489593    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489680    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489767    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.489776    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.490364    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.490407    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.499162    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.500484    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.500524    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.500549    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.500586    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 10:28:07.510517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.502305    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:07.590593 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.590513    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.591579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.591560    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.591662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.591594    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.591662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.591605    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.591662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.591630    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.599807 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.599782    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.599870 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.599817    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-36.ec2.internal\": node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.600956 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.600937    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"]
Jul 29 10:28:07.601011 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.601002    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.601964 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.601949    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.602019 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.601976    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.602019 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.601991    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.604430 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.604415    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.604951 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.604938    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.604998 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.604973    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.605205 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.605177    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.605205 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.605209    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.605351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.605220    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.605658 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.605644    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.605738 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.605672    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.605738 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.605685    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.607551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.607536    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.607618 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.607564    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 10:28:07.608250 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.608237    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 10:28:07.608314 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.608261    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 10:28:07.608314 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.608272    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeHasSufficientPID"
Jul 29 10:28:07.627700 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.627667    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.629255 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.629233    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-36.ec2.internal\" not found" node="ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.630012 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.629992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dba68a4d6621fead7294c30802645215-config\") pod \"kube-apiserver-proxy-ip-10-0-143-36.ec2.internal\" (UID: \"dba68a4d6621fead7294c30802645215\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.630096 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.630018    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/39dfc139e01d6335c83a263c684d8da3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal\" (UID: \"39dfc139e01d6335c83a263c684d8da3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.630096 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.630043    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39dfc139e01d6335c83a263c684d8da3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal\" (UID: \"39dfc139e01d6335c83a263c684d8da3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.633791 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.633772    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-36.ec2.internal\" not found" node="ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.728452 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.728417    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.730613 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.730593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dba68a4d6621fead7294c30802645215-config\") pod \"kube-apiserver-proxy-ip-10-0-143-36.ec2.internal\" (UID: \"dba68a4d6621fead7294c30802645215\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.730674 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.730620    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/39dfc139e01d6335c83a263c684d8da3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal\" (UID: \"39dfc139e01d6335c83a263c684d8da3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.730674 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.730636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39dfc139e01d6335c83a263c684d8da3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal\" (UID: \"39dfc139e01d6335c83a263c684d8da3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.730758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.730665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39dfc139e01d6335c83a263c684d8da3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal\" (UID: \"39dfc139e01d6335c83a263c684d8da3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.730758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.730691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dba68a4d6621fead7294c30802645215-config\") pod \"kube-apiserver-proxy-ip-10-0-143-36.ec2.internal\" (UID: \"dba68a4d6621fead7294c30802645215\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.730758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.730711    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/39dfc139e01d6335c83a263c684d8da3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal\" (UID: \"39dfc139e01d6335c83a263c684d8da3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.828838 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.828802    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.929580 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:07.929511    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:07.932786 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.932769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:07.939079 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:07.938868    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:08.030014 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.029969    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.130533 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.130503    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.230640 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.230556    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.231457 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.231437    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:08.331507 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.331467    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.355799 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.355778    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 10:28:08.356316 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.355935    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 10:28:08.356316 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.355973    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 10:28:08.432492 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.432452    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.451988 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.451952    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 10:23:08 +0000 UTC" deadline="2028-01-25 06:11:26.797755648 +0000 UTC"
Jul 29 10:28:08.451988 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.451987    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13075h43m18.345772155s"
Jul 29 10:28:08.533125 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.533088    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.633638 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.633602    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.649847 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:08.649810    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba68a4d6621fead7294c30802645215.slice/crio-ec23d8a73c61e3f80dc697876b28eff33ba05a4be0f64df996a1d2549797b2a1 WatchSource:0}: Error finding container ec23d8a73c61e3f80dc697876b28eff33ba05a4be0f64df996a1d2549797b2a1: Status 404 returned error can't find the container with id ec23d8a73c61e3f80dc697876b28eff33ba05a4be0f64df996a1d2549797b2a1
Jul 29 10:28:08.650080 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:08.650051    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39dfc139e01d6335c83a263c684d8da3.slice/crio-7cae729e78e593bd8d918309e10d7411d87cfe4c628fe509335c01ffc3f8d304 WatchSource:0}: Error finding container 7cae729e78e593bd8d918309e10d7411d87cfe4c628fe509335c01ffc3f8d304: Status 404 returned error can't find the container with id 7cae729e78e593bd8d918309e10d7411d87cfe4c628fe509335c01ffc3f8d304
Jul 29 10:28:08.657986 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.657965    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 10:28:08.734769 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:08.734724    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-36.ec2.internal\" not found"
Jul 29 10:28:08.783786 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.783723    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:08.807618 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.807586    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:08.829106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.829065    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:08.842965 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.842933    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 10:28:08.844070 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.844053    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal"
Jul 29 10:28:08.853351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:08.853313    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 10:28:09.406198 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.406165    2562 apiserver.go:52] "Watching apiserver"
Jul 29 10:28:09.413945 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.413918    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 10:28:09.414381 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.414359    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-x8m2t","openshift-multus/network-metrics-daemon-hrntc","openshift-network-diagnostics/network-check-target-mj9l6","openshift-network-operator/iptables-alerter-j7jgr","kube-system/global-pull-secret-syncer-xp6pm","kube-system/konnectivity-agent-dldrq","openshift-cluster-node-tuning-operator/tuned-z7jml","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal","openshift-multus/multus-wn858","openshift-ovn-kubernetes/ovnkube-node-4gwff","kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc","openshift-dns/node-resolver-vkshd","openshift-image-registry/node-ca-sx4n7"]
Jul 29 10:28:09.416937 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.416907    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.419177 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.419149    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.419673 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.419652    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-52mkr\""
Jul 29 10:28:09.421197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.419889    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.421197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.419893    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.421402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.421382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:09.421479 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.421460    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:09.422153 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.421787    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.422153 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.421835    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.422153 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.421888    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x4d6r\""
Jul 29 10:28:09.423708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.423689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.425928 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.425796    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.425928 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.425861    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:09.426310 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.426288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 10:28:09.426583 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.426559    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.426790 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.426775    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dd4qn\""
Jul 29 10:28:09.427021 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.427001    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.428040 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.428002    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.429310 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.428945    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 10:28:09.430312 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.430291    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 10:28:09.430422 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.430382    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zfqdm\""
Jul 29 10:28:09.430734 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.430715    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 10:28:09.431456 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.431407    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.433722 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.433703    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.434781 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.434373    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.434781 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.434438    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 10:28:09.434781 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.434559    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 10:28:09.434781 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.434779    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pvj7l\""
Jul 29 10:28:09.435010 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.434852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.436042 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.436022    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:09.436114 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.436092    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:09.437154 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.437129    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 10:28:09.437523 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.437500    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6wgb2\""
Jul 29 10:28:09.437766 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.437745    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 10:28:09.439517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-hostroot\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.439517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439357    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/35dae8d5-271b-4bac-837b-be006a50d274-multus-daemon-config\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.439517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439387    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-lib-modules\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.439517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjm8f\" (UniqueName: \"kubernetes.io/projected/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-kube-api-access-wjm8f\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.439517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439444    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/198702a2-64b9-4023-84f4-cb0388b57f62-kubelet-config\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.439517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439470    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c955947-80b8-42b8-8900-db6cc6c15b89-agent-certs\") pod \"konnectivity-agent-dldrq\" (UID: \"8c955947-80b8-42b8-8900-db6cc6c15b89\") " pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-k8s-cni-cncf-io\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439561    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-cni-multus\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-multus-certs\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439652    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439706    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysctl-conf\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-run\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.439856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-tmp\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.440188 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.439844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c329c2e8-06df-4cad-b6ea-5cff76ef5978-tmp-dir\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.440244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.440244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-cnibin\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.440481 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-cni-bin\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.440481 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysconfig\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.440481 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-tuned\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.440481 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440363    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-system-cni-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.440481 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440386    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 10:28:09.440481 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-var-lib-kubelet\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c955947-80b8-42b8-8900-db6cc6c15b89-konnectivity-ca\") pod \"konnectivity-agent-dldrq\" (UID: \"8c955947-80b8-42b8-8900-db6cc6c15b89\") " pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440523    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02a171d0-b0c0-4b86-90a2-4efabff92857-iptables-alerter-script\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a171d0-b0c0-4b86-90a2-4efabff92857-host-slash\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-cni-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440607    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-socket-dir-parent\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-kubernetes\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440703    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-host\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v56bk\" (UniqueName: \"kubernetes.io/projected/c329c2e8-06df-4cad-b6ea-5cff76ef5978-kube-api-access-v56bk\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.440761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/198702a2-64b9-4023-84f4-cb0388b57f62-dbus\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440790    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35dae8d5-271b-4bac-837b-be006a50d274-cni-binary-copy\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-modprobe-d\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-sys\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zx9x\" (UniqueName: \"kubernetes.io/projected/02a171d0-b0c0-4b86-90a2-4efabff92857-kube-api-access-9zx9x\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440961    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-systemd\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.440992    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-os-release\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-conf-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441069    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-etc-kubernetes\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441087    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441095    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs54\" (UniqueName: \"kubernetes.io/projected/35dae8d5-271b-4bac-837b-be006a50d274-kube-api-access-tcs54\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441129    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysctl-d\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.441167 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441161    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c329c2e8-06df-4cad-b6ea-5cff76ef5978-hosts-file\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.441833 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441192    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-netns\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.441833 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.441220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-kubelet\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.442074 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.442052    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bzm6b\""
Jul 29 10:28:09.442533 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.442505    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 10:28:09.442628 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.442553    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 10:28:09.442683 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.442511    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.442844 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.442822    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 10:28:09.443768 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.443745    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.443963 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.443941    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.444048 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.444004    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 10:28:09.444118 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.444103    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.444195 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.443947    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dzstd\""
Jul 29 10:28:09.444340 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.444309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 10:28:09.446860 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.445833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.450345 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.449432    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 10:28:09.450345 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.449497    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-f6gmd\""
Jul 29 10:28:09.450345 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.449645    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 10:28:09.450345 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.449694    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 10:28:09.472050 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.471792    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rx9xj"
Jul 29 10:28:09.480487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.480358    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rx9xj"
Jul 29 10:28:09.507021 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.506964    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal" event={"ID":"dba68a4d6621fead7294c30802645215","Type":"ContainerStarted","Data":"ec23d8a73c61e3f80dc697876b28eff33ba05a4be0f64df996a1d2549797b2a1"}
Jul 29 10:28:09.508670 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.508637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal" event={"ID":"39dfc139e01d6335c83a263c684d8da3","Type":"ContainerStarted","Data":"7cae729e78e593bd8d918309e10d7411d87cfe4c628fe509335c01ffc3f8d304"}
Jul 29 10:28:09.530380 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.530344    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjm8f\" (UniqueName: \"kubernetes.io/projected/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-kube-api-access-wjm8f\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-k8s-cni-cncf-io\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-multus-certs\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541817    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-node-log\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f22e8537-58dc-4d81-aee3-c978400cdc80-ovn-node-metrics-cert\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541871    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb49s\" (UniqueName: \"kubernetes.io/projected/79f1afb4-3c15-4858-8291-48c7603f443a-kube-api-access-nb49s\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541896    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-cni-bin\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-os-release\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7psfh\" (UniqueName: \"kubernetes.io/projected/abd0629b-e500-4831-b288-292c7a8c8c16-kube-api-access-7psfh\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.541992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysconfig\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-tuned\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542040    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-slash\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-env-overrides\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542087    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c955947-80b8-42b8-8900-db6cc6c15b89-konnectivity-ca\") pod \"konnectivity-agent-dldrq\" (UID: \"8c955947-80b8-42b8-8900-db6cc6c15b89\") " pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a171d0-b0c0-4b86-90a2-4efabff92857-host-slash\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542123    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-k8s-cni-cncf-io\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.544933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-cni-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a034c05-e7ae-4fd3-b69b-766314822683-host\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-multus-certs\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542205    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v56bk\" (UniqueName: \"kubernetes.io/projected/c329c2e8-06df-4cad-b6ea-5cff76ef5978-kube-api-access-v56bk\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542247    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/198702a2-64b9-4023-84f4-cb0388b57f62-dbus\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542262    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-cni-bin\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35dae8d5-271b-4bac-837b-be006a50d274-cni-binary-copy\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-sys\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542357    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysconfig\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-system-cni-dir\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542420    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-cni-binary-copy\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542444    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-run-netns\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-socket-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-os-release\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-conf-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.545921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542582    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-ovnkube-script-lib\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysctl-d\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/198702a2-64b9-4023-84f4-cb0388b57f62-dbus\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c329c2e8-06df-4cad-b6ea-5cff76ef5978-hosts-file\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-netns\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02a171d0-b0c0-4b86-90a2-4efabff92857-host-slash\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-log-socket\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542774    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-cni-netd\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542788    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-os-release\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.542945    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-conf-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c329c2e8-06df-4cad-b6ea-5cff76ef5978-hosts-file\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543192    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysctl-d\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-cni-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-sys\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543370    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/35dae8d5-271b-4bac-837b-be006a50d274-cni-binary-copy\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543373    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-device-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.550251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-lib-modules\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/198702a2-64b9-4023-84f4-cb0388b57f62-kubelet-config\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c955947-80b8-42b8-8900-db6cc6c15b89-agent-certs\") pod \"konnectivity-agent-dldrq\" (UID: \"8c955947-80b8-42b8-8900-db6cc6c15b89\") " pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-cni-multus\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-var-lib-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543577    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-etc-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysctl-conf\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543624    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-run\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-tmp\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-cni-multus\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c329c2e8-06df-4cad-b6ea-5cff76ef5978-tmp-dir\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543748    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-cnibin\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543801    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8c955947-80b8-42b8-8900-db6cc6c15b89-konnectivity-ca\") pod \"konnectivity-agent-dldrq\" (UID: \"8c955947-80b8-42b8-8900-db6cc6c15b89\") " pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdkcs\" (UniqueName: \"kubernetes.io/projected/f22e8537-58dc-4d81-aee3-c978400cdc80-kube-api-access-rdkcs\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a034c05-e7ae-4fd3-b69b-766314822683-serviceca\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543876    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-system-cni-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.551093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-systemd\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543945    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-ovn\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.543973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-registration-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544000    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds9fc\" (UniqueName: \"kubernetes.io/projected/4ab4e177-d475-40fa-88d6-226c2604749c-kube-api-access-ds9fc\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544098    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c329c2e8-06df-4cad-b6ea-5cff76ef5978-tmp-dir\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544143    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9zt5\" (UniqueName: \"kubernetes.io/projected/0a034c05-e7ae-4fd3-b69b-766314822683-kube-api-access-d9zt5\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-var-lib-kubelet\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02a171d0-b0c0-4b86-90a2-4efabff92857-iptables-alerter-script\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-socket-dir-parent\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-sysctl-conf\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-kubernetes\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-host\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544425    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-host\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544431    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-run\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-run-ovn-kubernetes\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544519    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-cni-bin\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.551850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-modprobe-d\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.544586    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zx9x\" (UniqueName: \"kubernetes.io/projected/02a171d0-b0c0-4b86-90a2-4efabff92857-kube-api-access-9zx9x\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544624    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-kubelet\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.544662    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:10.044641241 +0000 UTC m=+3.033490691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-systemd\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-run-netns\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-etc-kubernetes\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs54\" (UniqueName: \"kubernetes.io/projected/35dae8d5-271b-4bac-837b-be006a50d274-kube-api-access-tcs54\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-systemd-units\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544828    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-lib-modules\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544844    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-ovnkube-config\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/198702a2-64b9-4023-84f4-cb0388b57f62-kubelet-config\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.544911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-sys-fs\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:09.552633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545037    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-kubelet\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-hostroot\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545222    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-modprobe-d\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/35dae8d5-271b-4bac-837b-be006a50d274-multus-daemon-config\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-cnibin\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-systemd\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-hostroot\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-kubernetes\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-multus-socket-dir-parent\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-cnibin\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-etc-kubernetes\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-host-var-lib-kubelet\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-var-lib-kubelet\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.545744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/35dae8d5-271b-4bac-837b-be006a50d274-system-cni-dir\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.546233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/35dae8d5-271b-4bac-837b-be006a50d274-multus-daemon-config\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.553407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.546245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/02a171d0-b0c0-4b86-90a2-4efabff92857-iptables-alerter-script\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.547086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-etc-tuned\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.547942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8c955947-80b8-42b8-8900-db6cc6c15b89-agent-certs\") pod \"konnectivity-agent-dldrq\" (UID: \"8c955947-80b8-42b8-8900-db6cc6c15b89\") " pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.549471    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.549495    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.549509    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.549581    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:10.049562394 +0000 UTC m=+3.038411850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:09.554289 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.550816    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-tmp\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.555112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.555080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjm8f\" (UniqueName: \"kubernetes.io/projected/f4711797-71ff-473d-8ba2-2dfb6ecfa9e2-kube-api-access-wjm8f\") pod \"tuned-z7jml\" (UID: \"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2\") " pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.555995 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.555955    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zx9x\" (UniqueName: \"kubernetes.io/projected/02a171d0-b0c0-4b86-90a2-4efabff92857-kube-api-access-9zx9x\") pod \"iptables-alerter-j7jgr\" (UID: \"02a171d0-b0c0-4b86-90a2-4efabff92857\") " pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.557551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.557374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs54\" (UniqueName: \"kubernetes.io/projected/35dae8d5-271b-4bac-837b-be006a50d274-kube-api-access-tcs54\") pod \"multus-wn858\" (UID: \"35dae8d5-271b-4bac-837b-be006a50d274\") " pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.558755 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.558730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v56bk\" (UniqueName: \"kubernetes.io/projected/c329c2e8-06df-4cad-b6ea-5cff76ef5978-kube-api-access-v56bk\") pod \"node-resolver-vkshd\" (UID: \"c329c2e8-06df-4cad-b6ea-5cff76ef5978\") " pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.646031 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.645961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-log-socket\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-cni-netd\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646075    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-device-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-var-lib-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-etc-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdkcs\" (UniqueName: \"kubernetes.io/projected/f22e8537-58dc-4d81-aee3-c978400cdc80-kube-api-access-rdkcs\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a034c05-e7ae-4fd3-b69b-766314822683-serviceca\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-systemd\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-ovn\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-registration-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds9fc\" (UniqueName: \"kubernetes.io/projected/4ab4e177-d475-40fa-88d6-226c2604749c-kube-api-access-ds9fc\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9zt5\" (UniqueName: \"kubernetes.io/projected/0a034c05-e7ae-4fd3-b69b-766314822683-kube-api-access-d9zt5\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646313    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646344    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-run-ovn-kubernetes\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-cni-bin\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-kubelet\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-systemd-units\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-ovnkube-config\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-sys-fs\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646480    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.646655 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646582    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-cnibin\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646609    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-node-log\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f22e8537-58dc-4d81-aee3-c978400cdc80-ovn-node-metrics-cert\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb49s\" (UniqueName: \"kubernetes.io/projected/79f1afb4-3c15-4858-8291-48c7603f443a-kube-api-access-nb49s\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-os-release\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7psfh\" (UniqueName: \"kubernetes.io/projected/abd0629b-e500-4831-b288-292c7a8c8c16-kube-api-access-7psfh\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-slash\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-env-overrides\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a034c05-e7ae-4fd3-b69b-766314822683-host\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-system-cni-dir\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-cni-binary-copy\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-run-netns\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-socket-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.646980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.647467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-ovnkube-script-lib\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-ovnkube-script-lib\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-log-socket\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-cni-netd\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647870    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-device-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-var-lib-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.647950    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-etc-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648560 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648376    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.648705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0a034c05-e7ae-4fd3-b69b-766314822683-serviceca\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.648764 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648734    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-systemd\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648764 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648753    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.648862 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648761    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-ovn\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.648862 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-registration-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.648862 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648835    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.648862 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-run-ovn-kubernetes\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649041 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-cni-bin\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649041 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648909    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-kubelet\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649041 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.648933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-systemd-units\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649041 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-cnibin\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.649237 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-node-log\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649237 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-ovnkube-config\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.649351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649307    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-sys-fs\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.649459 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.649379    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:09.649459 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0a034c05-e7ae-4fd3-b69b-766314822683-host\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.649459 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:09.649447    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:10.149430185 +0000 UTC m=+3.138279633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:09.649664 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-os-release\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.649723 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-run-netns\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.649723 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/abd0629b-e500-4831-b288-292c7a8c8c16-system-cni-dir\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.649881 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.649861    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-host-slash\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.650194 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.650174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-cni-binary-copy\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.650257 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.650207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f22e8537-58dc-4d81-aee3-c978400cdc80-env-overrides\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.650257 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.650251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f22e8537-58dc-4d81-aee3-c978400cdc80-run-openvswitch\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.650369 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.650296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4ab4e177-d475-40fa-88d6-226c2604749c-socket-dir\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.650812 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.650769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/abd0629b-e500-4831-b288-292c7a8c8c16-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.651888 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.651866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f22e8537-58dc-4d81-aee3-c978400cdc80-ovn-node-metrics-cert\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.660249 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.660162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds9fc\" (UniqueName: \"kubernetes.io/projected/4ab4e177-d475-40fa-88d6-226c2604749c-kube-api-access-ds9fc\") pod \"aws-ebs-csi-driver-node-w8cxc\" (UID: \"4ab4e177-d475-40fa-88d6-226c2604749c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:09.662243 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.662182    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7psfh\" (UniqueName: \"kubernetes.io/projected/abd0629b-e500-4831-b288-292c7a8c8c16-kube-api-access-7psfh\") pod \"multus-additional-cni-plugins-x8m2t\" (UID: \"abd0629b-e500-4831-b288-292c7a8c8c16\") " pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.662402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.662343    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb49s\" (UniqueName: \"kubernetes.io/projected/79f1afb4-3c15-4858-8291-48c7603f443a-kube-api-access-nb49s\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:09.662570 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.662546    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdkcs\" (UniqueName: \"kubernetes.io/projected/f22e8537-58dc-4d81-aee3-c978400cdc80-kube-api-access-rdkcs\") pod \"ovnkube-node-4gwff\" (UID: \"f22e8537-58dc-4d81-aee3-c978400cdc80\") " pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.663387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.663342    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9zt5\" (UniqueName: \"kubernetes.io/projected/0a034c05-e7ae-4fd3-b69b-766314822683-kube-api-access-d9zt5\") pod \"node-ca-sx4n7\" (UID: \"0a034c05-e7ae-4fd3-b69b-766314822683\") " pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.731241 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.731200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z7jml"
Jul 29 10:28:09.741223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.741183    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vkshd"
Jul 29 10:28:09.754236 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.754206    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j7jgr"
Jul 29 10:28:09.761344 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.761286    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:09.769616 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.769588    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wn858"
Jul 29 10:28:09.782692 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.782665    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x8m2t"
Jul 29 10:28:09.788068 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.788042    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:09.791721 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.791693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:09.800856 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.800827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sx4n7"
Jul 29 10:28:09.808083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:09.808054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc"
Jul 29 10:28:10.050184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:10.050077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:10.050184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:10.050136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:10.050417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.050244    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:10.050417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.050262    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:10.050417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.050280    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:10.050417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.050294    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:10.050417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.050319    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:11.050300256 +0000 UTC m=+4.039149714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:10.050417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.050354    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:11.050325561 +0000 UTC m=+4.039175019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:10.151054 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:10.151011    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:10.151279 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.151202    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:10.151381 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:10.151292    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:11.151269206 +0000 UTC m=+4.140118658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:10.483526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:10.483437    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 10:23:10 +0000 UTC" deadline="2028-03-01 06:25:55.909949304 +0000 UTC"
Jul 29 10:28:10.483526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:10.483470    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13939h57m45.426482881s"
Jul 29 10:28:10.842388 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.842352    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4711797_71ff_473d_8ba2_2dfb6ecfa9e2.slice/crio-b98ff371a1b0f018a210e09c39c2b8cf0914e186f447eac23afbd28506bb2717 WatchSource:0}: Error finding container b98ff371a1b0f018a210e09c39c2b8cf0914e186f447eac23afbd28506bb2717: Status 404 returned error can't find the container with id b98ff371a1b0f018a210e09c39c2b8cf0914e186f447eac23afbd28506bb2717
Jul 29 10:28:10.844064 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.844028    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a171d0_b0c0_4b86_90a2_4efabff92857.slice/crio-3967656dd809608acecec41f0972f6ceeb9c3ef176293899936a6b4e4b6c5705 WatchSource:0}: Error finding container 3967656dd809608acecec41f0972f6ceeb9c3ef176293899936a6b4e4b6c5705: Status 404 returned error can't find the container with id 3967656dd809608acecec41f0972f6ceeb9c3ef176293899936a6b4e4b6c5705
Jul 29 10:28:10.868677 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.868641    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc329c2e8_06df_4cad_b6ea_5cff76ef5978.slice/crio-acc531fe9970bdac609b70018e773d1d11a46ab93676ed206744bdfccf89af10 WatchSource:0}: Error finding container acc531fe9970bdac609b70018e773d1d11a46ab93676ed206744bdfccf89af10: Status 404 returned error can't find the container with id acc531fe9970bdac609b70018e773d1d11a46ab93676ed206744bdfccf89af10
Jul 29 10:28:10.873557 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.873525    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35dae8d5_271b_4bac_837b_be006a50d274.slice/crio-c52d7ace903cc044162d7422722954fbd0baa4de1a422eae5a837c0ab31a55d7 WatchSource:0}: Error finding container c52d7ace903cc044162d7422722954fbd0baa4de1a422eae5a837c0ab31a55d7: Status 404 returned error can't find the container with id c52d7ace903cc044162d7422722954fbd0baa4de1a422eae5a837c0ab31a55d7
Jul 29 10:28:10.875250 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.875218    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a034c05_e7ae_4fd3_b69b_766314822683.slice/crio-e76d964a6e33509162f8d42d32856f9b1fcb54aa73e5250f5a9591200e9139c8 WatchSource:0}: Error finding container e76d964a6e33509162f8d42d32856f9b1fcb54aa73e5250f5a9591200e9139c8: Status 404 returned error can't find the container with id e76d964a6e33509162f8d42d32856f9b1fcb54aa73e5250f5a9591200e9139c8
Jul 29 10:28:10.876686 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.876624    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22e8537_58dc_4d81_aee3_c978400cdc80.slice/crio-60bffd1b075f98156b7c40b5d11c0594bbd42880af238a2d906f94f76f8ee6dd WatchSource:0}: Error finding container 60bffd1b075f98156b7c40b5d11c0594bbd42880af238a2d906f94f76f8ee6dd: Status 404 returned error can't find the container with id 60bffd1b075f98156b7c40b5d11c0594bbd42880af238a2d906f94f76f8ee6dd
Jul 29 10:28:10.877707 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.877673    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab4e177_d475_40fa_88d6_226c2604749c.slice/crio-f73933e38537776fd7e5e33e61dc485c694de732e4dd45e48782fb76f0d140ca WatchSource:0}: Error finding container f73933e38537776fd7e5e33e61dc485c694de732e4dd45e48782fb76f0d140ca: Status 404 returned error can't find the container with id f73933e38537776fd7e5e33e61dc485c694de732e4dd45e48782fb76f0d140ca
Jul 29 10:28:10.879516 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.879492    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c955947_80b8_42b8_8900_db6cc6c15b89.slice/crio-6fcab51923a3fd9eefdc9c8d824ab300d9b9ccccb837dea6ce6e99e2eff5a9e7 WatchSource:0}: Error finding container 6fcab51923a3fd9eefdc9c8d824ab300d9b9ccccb837dea6ce6e99e2eff5a9e7: Status 404 returned error can't find the container with id 6fcab51923a3fd9eefdc9c8d824ab300d9b9ccccb837dea6ce6e99e2eff5a9e7
Jul 29 10:28:10.881526 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:10.881505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd0629b_e500_4831_b288_292c7a8c8c16.slice/crio-94f33d432ddb32c5059bc7c794de2cb36f81e33dc7cf7b68303e025062feadc8 WatchSource:0}: Error finding container 94f33d432ddb32c5059bc7c794de2cb36f81e33dc7cf7b68303e025062feadc8: Status 404 returned error can't find the container with id 94f33d432ddb32c5059bc7c794de2cb36f81e33dc7cf7b68303e025062feadc8
Jul 29 10:28:11.057618 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.057411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:11.057804 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.057626    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:11.057804 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.057654    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:11.057804 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.057654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:11.057804 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.057668    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:11.057804 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.057731    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:13.057710724 +0000 UTC m=+6.046560176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:11.057980 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.057806    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:11.057980 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.057860    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:13.057847304 +0000 UTC m=+6.046696744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:11.158448 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.158411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:11.158616 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.158594    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:11.158688 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.158679    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:13.15865709 +0000 UTC m=+6.147506543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:11.389782 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.389385    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 10:28:11.484579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.484475    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 10:23:10 +0000 UTC" deadline="2028-01-27 18:06:28.092071857 +0000 UTC"
Jul 29 10:28:11.484579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.484538    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13135h38m16.607538105s"
Jul 29 10:28:11.507022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.505678    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:11.507022 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.505805    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:11.507022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.506269    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:11.507022 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.506374    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:11.507022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.506568    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:11.507022 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:11.506656    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:11.537436 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.537169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal" event={"ID":"dba68a4d6621fead7294c30802645215","Type":"ContainerStarted","Data":"1381c9e454a6ac8af8db41b55f15f30c461c03ceecd7674d4d39ade5ef8bc21a"}
Jul 29 10:28:11.540152 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.540069    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerStarted","Data":"94f33d432ddb32c5059bc7c794de2cb36f81e33dc7cf7b68303e025062feadc8"}
Jul 29 10:28:11.542984 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.542952    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dldrq" event={"ID":"8c955947-80b8-42b8-8900-db6cc6c15b89","Type":"ContainerStarted","Data":"6fcab51923a3fd9eefdc9c8d824ab300d9b9ccccb837dea6ce6e99e2eff5a9e7"}
Jul 29 10:28:11.548708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.548677    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"60bffd1b075f98156b7c40b5d11c0594bbd42880af238a2d906f94f76f8ee6dd"}
Jul 29 10:28:11.552472 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.552435    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vkshd" event={"ID":"c329c2e8-06df-4cad-b6ea-5cff76ef5978","Type":"ContainerStarted","Data":"acc531fe9970bdac609b70018e773d1d11a46ab93676ed206744bdfccf89af10"}
Jul 29 10:28:11.554351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.554294    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-36.ec2.internal" podStartSLOduration=2.554274979 podStartE2EDuration="2.554274979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:11.553690733 +0000 UTC m=+4.542540196" watchObservedRunningTime="2026-07-29 10:28:11.554274979 +0000 UTC m=+4.543124438"
Jul 29 10:28:11.557706 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.557654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z7jml" event={"ID":"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2","Type":"ContainerStarted","Data":"b98ff371a1b0f018a210e09c39c2b8cf0914e186f447eac23afbd28506bb2717"}
Jul 29 10:28:11.563204 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.563178    2562 generic.go:356] "Generic (PLEG): container finished" podID="39dfc139e01d6335c83a263c684d8da3" containerID="240dd07fff3bd42afdf407c74c72b658d51e91339a37a86316795444f49a4aa0" exitCode=0
Jul 29 10:28:11.563313 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.563256    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal" event={"ID":"39dfc139e01d6335c83a263c684d8da3","Type":"ContainerDied","Data":"240dd07fff3bd42afdf407c74c72b658d51e91339a37a86316795444f49a4aa0"}
Jul 29 10:28:11.568869 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.568842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sx4n7" event={"ID":"0a034c05-e7ae-4fd3-b69b-766314822683","Type":"ContainerStarted","Data":"e76d964a6e33509162f8d42d32856f9b1fcb54aa73e5250f5a9591200e9139c8"}
Jul 29 10:28:11.573705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.573622    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc" event={"ID":"4ab4e177-d475-40fa-88d6-226c2604749c","Type":"ContainerStarted","Data":"f73933e38537776fd7e5e33e61dc485c694de732e4dd45e48782fb76f0d140ca"}
Jul 29 10:28:11.578279 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.578115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wn858" event={"ID":"35dae8d5-271b-4bac-837b-be006a50d274","Type":"ContainerStarted","Data":"c52d7ace903cc044162d7422722954fbd0baa4de1a422eae5a837c0ab31a55d7"}
Jul 29 10:28:11.584564 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:11.584291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j7jgr" event={"ID":"02a171d0-b0c0-4b86-90a2-4efabff92857","Type":"ContainerStarted","Data":"3967656dd809608acecec41f0972f6ceeb9c3ef176293899936a6b4e4b6c5705"}
Jul 29 10:28:12.603523 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:12.603446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal" event={"ID":"39dfc139e01d6335c83a263c684d8da3","Type":"ContainerStarted","Data":"f8436afabbf6d55e5b2df09f035d1f3d13830546ce4859e477777ccd6175935f"}
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:13.073751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:13.073815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.074030    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.074105    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:17.07408564 +0000 UTC m=+10.062935082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.074558    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.074582    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.074598    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:13.074739 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.074645    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:17.074629573 +0000 UTC m=+10.063479012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:13.175236 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:13.175177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:13.175474 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.175445    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:13.175544 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.175515    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:17.175494669 +0000 UTC m=+10.164344111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:13.501777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:13.501662    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:13.501969 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.501798    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:13.502250 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:13.502227    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:13.502417 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.502374    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:13.502518 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:13.502469    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:13.502571 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:13.502549    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:15.501941 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:15.501893    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:15.502446 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:15.502040    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:15.502446 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:15.501894    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:15.502446 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:15.502145    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:15.502446 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:15.502167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:15.502446 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:15.502238    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:17.108557 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:17.108503    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:17.108572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.108733    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.108809    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:25.108785959 +0000 UTC m=+18.097635401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.108732    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.108859    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.108877    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:17.109060 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.108935    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:25.108915498 +0000 UTC m=+18.097764938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:17.209387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:17.209330    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:17.209587 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.209515    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:17.209685 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.209605    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:25.209582515 +0000 UTC m=+18.198431954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:17.501731 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:17.501647    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:17.501891 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.501783    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:17.501891 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:17.501795    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:17.502016 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:17.501915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:17.502016 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.501966    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:17.502016 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:17.501991    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:19.501822 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:19.501781    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:19.502430 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:19.501787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:19.502430 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:19.501977    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:19.502430 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:19.502043    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:19.502430 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:19.501793    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:19.502430 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:19.502141    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:21.501565 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:21.501531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:21.501565 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:21.501547    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:21.502085 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:21.501664    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:21.502085 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:21.501690    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:21.502085 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:21.501785    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:21.502085 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:21.501907    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:23.501514 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:23.501474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:23.501957 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:23.501474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:23.501957 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:23.501611    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:23.501957 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:23.501474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:23.501957 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:23.501698    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:23.501957 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:23.501754    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:25.172798 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:25.172759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:25.172817    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.172918    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.172937    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.172956    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.172968    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.172989    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:41.172970104 +0000 UTC m=+34.161819557 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:25.173259 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.173008    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:41.172998611 +0000 UTC m=+34.161848049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:25.273633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:25.273589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:25.273857 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.273719    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:25.273857 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.273787    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:28:41.273769121 +0000 UTC m=+34.262618561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:25.501847 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:25.501761    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:25.502011 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:25.501775    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:25.502011 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:25.501896    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:25.502011 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.501891    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:25.502011 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.501991    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:25.502189 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:25.502081    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:27.501586 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:27.501549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:27.502038 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:27.501654    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:27.502038 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:27.501663    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:27.502038 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:27.501691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:27.502038 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:27.501770    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:27.502038 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:27.501860    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:29.501662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:29.501621    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:29.502272 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:29.501622    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:29.502272 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:29.501763    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:29.502272 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:29.501783    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:29.502272 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:29.501883    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:29.502272 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:29.501956    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:31.501641 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.501226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:31.502383 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.501237    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:31.502383 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:31.501672    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:31.502383 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.501289    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:31.502383 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:31.501757    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:31.502383 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:31.501862    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:31.667281 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.667249    2562 generic.go:356] "Generic (PLEG): container finished" podID="abd0629b-e500-4831-b288-292c7a8c8c16" containerID="07a4016bb30f51b598cc7d7cea1c71cb86fb09c3b14bdea7ffd19ee4adc4e263" exitCode=0
Jul 29 10:28:31.667489 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.667356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerDied","Data":"07a4016bb30f51b598cc7d7cea1c71cb86fb09c3b14bdea7ffd19ee4adc4e263"}
Jul 29 10:28:31.668833 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.668809    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dldrq" event={"ID":"8c955947-80b8-42b8-8900-db6cc6c15b89","Type":"ContainerStarted","Data":"73f41a75670ed981567c505a55d9322477ad8f6abf65038120a4ffcf38a46355"}
Jul 29 10:28:31.671822 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.671788    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"8f054a6b5f8665eef8c16cf3d0e0ac2697b369f6ca0b11014081d7467db507b0"}
Jul 29 10:28:31.671925 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.671870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"38596fc499a7b058bf1067a743335d9a5504940c2dce8309ea11b956013faddb"}
Jul 29 10:28:31.671983 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.671923    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"68fbedbd07cc8b12392de6c9f1d14bd4299bd23c963c80e4ca55cfb8e980a7fe"}
Jul 29 10:28:31.671983 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.671971    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"6e6283377b7e29ee81d12dec4cf9a837dc0e3b0226935685d52021162a30e1d1"}
Jul 29 10:28:31.672072 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.672012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"22789238eb92a9674c3b4b5a5af03a06d5f0ab24e52d9b271a604253b343504f"}
Jul 29 10:28:31.672072 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.672049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"80eb48e70be28660fa72412766e190dac1bba1c4e9f97c87d8f1fe120505eba2"}
Jul 29 10:28:31.673199 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.673127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vkshd" event={"ID":"c329c2e8-06df-4cad-b6ea-5cff76ef5978","Type":"ContainerStarted","Data":"267be09320284a940b44c2054c6b1a1faa3fdce40f1bcf733622b779beefda0f"}
Jul 29 10:28:31.674644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.674617    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z7jml" event={"ID":"f4711797-71ff-473d-8ba2-2dfb6ecfa9e2","Type":"ContainerStarted","Data":"176403ad068ce43d1fc690a2ea2b2bc214aa958cacc5bd78794d849863d7ffc4"}
Jul 29 10:28:31.675882 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.675862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sx4n7" event={"ID":"0a034c05-e7ae-4fd3-b69b-766314822683","Type":"ContainerStarted","Data":"71cc8af66cbfa4b5e959c1a7d65d56c1e97c31a5779976bded3bdfa0cca46a94"}
Jul 29 10:28:31.677104 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.677087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc" event={"ID":"4ab4e177-d475-40fa-88d6-226c2604749c","Type":"ContainerStarted","Data":"7d541ff72c0ff6f6bd9d4ee2f494e68eb8efa730b3f43a4f9694a2f4c87e4023"}
Jul 29 10:28:31.678533 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.678512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wn858" event={"ID":"35dae8d5-271b-4bac-837b-be006a50d274","Type":"ContainerStarted","Data":"717c76b2cef30583504130202c5d5f74175d821b530563f9729f56b056284f15"}
Jul 29 10:28:31.691963 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.691914    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-36.ec2.internal" podStartSLOduration=22.691899683 podStartE2EDuration="22.691899683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:12.623045325 +0000 UTC m=+5.611894785" watchObservedRunningTime="2026-07-29 10:28:31.691899683 +0000 UTC m=+24.680749139"
Jul 29 10:28:31.707698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.707649    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dldrq" podStartSLOduration=7.909874541 podStartE2EDuration="23.707631662s" totalImagesPullingTime="15.797757121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:31.707418028 +0000 UTC m=+24.696267483" watchObservedRunningTime="2026-07-29 10:28:31.707631662 +0000 UTC m=+24.696481117"
Jul 29 10:28:31.722951 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.722894    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vkshd" podStartSLOduration=4.052791548 podStartE2EDuration="23.722877877s" totalImagesPullingTime="19.670086329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:31.722705796 +0000 UTC m=+24.711555251" watchObservedRunningTime="2026-07-29 10:28:31.722877877 +0000 UTC m=+24.711727335"
Jul 29 10:28:31.741406 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.741351    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wn858" podStartSLOduration=4.011087193 podStartE2EDuration="23.741338156s" totalImagesPullingTime="19.730250963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:31.741089809 +0000 UTC m=+24.729939265" watchObservedRunningTime="2026-07-29 10:28:31.741338156 +0000 UTC m=+24.730187605"
Jul 29 10:28:31.756183 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.756093    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sx4n7" podStartSLOduration=4.091972794 podStartE2EDuration="23.756076986s" totalImagesPullingTime="19.664104192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:31.75590812 +0000 UTC m=+24.744757577" watchObservedRunningTime="2026-07-29 10:28:31.756076986 +0000 UTC m=+24.744926442"
Jul 29 10:28:31.773500 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:31.773425    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z7jml" podStartSLOduration=4.095858568 podStartE2EDuration="23.773387459s" totalImagesPullingTime="19.677528891s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:31.773287211 +0000 UTC m=+24.762136668" watchObservedRunningTime="2026-07-29 10:28:31.773387459 +0000 UTC m=+24.762236973"
Jul 29 10:28:32.150449 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.150421    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 10:28:32.503556 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.503411    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T10:28:32.150440794Z","UUID":"f5523a45-2494-4e9f-b709-397c58329658","Handler":null,"Name":"","Endpoint":""}
Jul 29 10:28:32.505139 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.505115    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 10:28:32.505262 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.505151    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 10:28:32.684523 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.684486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc" event={"ID":"4ab4e177-d475-40fa-88d6-226c2604749c","Type":"ContainerStarted","Data":"d592b289441f9063460fe68fb9a950613846c686fc998c0239827a1fab861a23"}
Jul 29 10:28:32.686336 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.686288    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j7jgr" event={"ID":"02a171d0-b0c0-4b86-90a2-4efabff92857","Type":"ContainerStarted","Data":"42222a9d69a3c8dddbb029f661a08c8d3d3e786da59785e7d62ef9c5bbf991e8"}
Jul 29 10:28:32.708201 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:32.708141    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j7jgr" podStartSLOduration=8.894674377 podStartE2EDuration="24.708118847s" totalImagesPullingTime="15.81344447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:32.707489127 +0000 UTC m=+25.696338583" watchObservedRunningTime="2026-07-29 10:28:32.708118847 +0000 UTC m=+25.696968304"
Jul 29 10:28:33.501588 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:33.501555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:33.501766 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:33.501599    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:33.501766 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:33.501555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:33.501766 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:33.501694    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:33.501766 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:33.501759    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:33.501977 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:33.501824    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:33.691169 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:33.691132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc" event={"ID":"4ab4e177-d475-40fa-88d6-226c2604749c","Type":"ContainerStarted","Data":"6ef82335f00e7fb4c35040ee873f26953789c86f9b7052ca3262f615ff9b8207"}
Jul 29 10:28:33.695857 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:33.695674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"4629114bf1b2d7aca37ef026271b429276cdc448c95b8e8ecb91ff7d640b87f3"}
Jul 29 10:28:35.501667 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:35.501586    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:35.502203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:35.501585    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:35.502203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:35.501600    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:35.502203 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:35.501731    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:35.502203 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:35.501845    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:35.502203 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:35.501941    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:36.525898 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.525623    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:36.526667 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.526230    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:36.541357 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.541286    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8cxc" podStartSLOduration=6.295407673 podStartE2EDuration="28.54126606s" totalImagesPullingTime="22.245858387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:33.70931473 +0000 UTC m=+26.698164176" watchObservedRunningTime="2026-07-29 10:28:36.54126606 +0000 UTC m=+29.530115519"
Jul 29 10:28:36.704619 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.704584    2562 generic.go:356] "Generic (PLEG): container finished" podID="abd0629b-e500-4831-b288-292c7a8c8c16" containerID="1c2ce9a72e890a533dd2a2332fef6bc69534fd44711d6292109468fad6a9eb2e" exitCode=0
Jul 29 10:28:36.704829 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.704670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerDied","Data":"1c2ce9a72e890a533dd2a2332fef6bc69534fd44711d6292109468fad6a9eb2e"}
Jul 29 10:28:36.708716 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.708684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" event={"ID":"f22e8537-58dc-4d81-aee3-c978400cdc80","Type":"ContainerStarted","Data":"fd2120a685f32342723b418392ff573ab01f5a744be42e99af39a5440e8a41b2"}
Jul 29 10:28:36.708937 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.708914    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:36.709390 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.709373    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dldrq"
Jul 29 10:28:36.773184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:36.773128    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff" podStartSLOduration=9.051297047 podStartE2EDuration="28.773112026s" totalImagesPullingTime="19.721814979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:36.772194514 +0000 UTC m=+29.761043960" watchObservedRunningTime="2026-07-29 10:28:36.773112026 +0000 UTC m=+29.761961481"
Jul 29 10:28:37.501858 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.501823    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:37.502135 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:37.501927    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:37.502135 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.501972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:37.502135 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:37.502088    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:37.502306 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.502135    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:37.502306 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:37.502246    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:37.712182 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.712146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:37.712657 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.712235    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:37.712657 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.712285    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:37.728107 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.728074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:37.728294 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:37.728278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:28:38.266747 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.266656    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hrntc"]
Jul 29 10:28:38.266901 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.266819    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:38.266970 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:38.266939    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:38.269299 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.269271    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xp6pm"]
Jul 29 10:28:38.269464 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.269393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:38.269517 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:38.269474    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:38.272134 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.272105    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mj9l6"]
Jul 29 10:28:38.272272 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.272226    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:38.272365 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:38.272339    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:38.716267 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.716195    2562 generic.go:356] "Generic (PLEG): container finished" podID="abd0629b-e500-4831-b288-292c7a8c8c16" containerID="0cebcfbc109bd23473803e8abae2d445f1e14e30e466b88f1c769e26e9f3eb8e" exitCode=0
Jul 29 10:28:38.716731 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:38.716281    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerDied","Data":"0cebcfbc109bd23473803e8abae2d445f1e14e30e466b88f1c769e26e9f3eb8e"}
Jul 29 10:28:39.501022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:39.500792    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:39.501215 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:39.500808    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:39.501215 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:39.501115    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:39.501345 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:39.501244    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:39.501345 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:39.500870    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:39.501445 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:39.501365    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:40.723780 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:40.723747    2562 generic.go:356] "Generic (PLEG): container finished" podID="abd0629b-e500-4831-b288-292c7a8c8c16" containerID="a26b19e619154f1f6af9969e0513ae8f7f30da7922f50cb77f5ce0673d8ab7ef" exitCode=0
Jul 29 10:28:40.724300 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:40.723798    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerDied","Data":"a26b19e619154f1f6af9969e0513ae8f7f30da7922f50cb77f5ce0673d8ab7ef"}
Jul 29 10:28:41.182461 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:41.182372    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:41.182613 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:41.182465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:41.182613 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.182555    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:41.182613 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.182587    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 10:28:41.182613 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.182602    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 10:28:41.182613 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.182611    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hl9dl for pod openshift-network-diagnostics/network-check-target-mj9l6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:41.182832 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.182640    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret podName:198702a2-64b9-4023-84f4-cb0388b57f62 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:13.182616962 +0000 UTC m=+66.171466399 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret") pod "global-pull-secret-syncer-xp6pm" (UID: "198702a2-64b9-4023-84f4-cb0388b57f62") : object "kube-system"/"original-pull-secret" not registered
Jul 29 10:28:41.182832 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.182661    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl podName:a2c68518-a780-4462-947d-f73790788f23 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:13.182652043 +0000 UTC m=+66.171501480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hl9dl" (UniqueName: "kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl") pod "network-check-target-mj9l6" (UID: "a2c68518-a780-4462-947d-f73790788f23") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 10:28:41.283627 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:41.283590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:41.283801 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.283762    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:41.283852 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.283839    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:29:13.283819123 +0000 UTC m=+66.272668561 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 10:28:41.501644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:41.501563    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:41.501644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:41.501598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:41.501831 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:41.501577    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:41.501831 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.501688    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xp6pm" podUID="198702a2-64b9-4023-84f4-cb0388b57f62"
Jul 29 10:28:41.501831 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.501761    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrntc" podUID="79f1afb4-3c15-4858-8291-48c7603f443a"
Jul 29 10:28:41.501956 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:41.501847    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mj9l6" podUID="a2c68518-a780-4462-947d-f73790788f23"
Jul 29 10:28:43.350773 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.350690    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-36.ec2.internal" event="NodeReady"
Jul 29 10:28:43.351387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.350856    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 10:28:43.386930 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.386896    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c48fd7547-8z9mq"]
Jul 29 10:28:43.391870 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.391844    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.395112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.394946    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 10:28:43.395112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.394966    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 10:28:43.395112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.394977    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 10:28:43.395112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.395112    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6wkgt\""
Jul 29 10:28:43.400231 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.399773    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 10:28:43.403973 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.403944    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zbcld"]
Jul 29 10:28:43.407821 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.407794    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r7d54"]
Jul 29 10:28:43.408003 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.407986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.411661 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.411623    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rxxsv\""
Jul 29 10:28:43.411802 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.411733    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.411945 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.411928    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 10:28:43.412442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.411643    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c48fd7547-8z9mq"]
Jul 29 10:28:43.413139 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.411051    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 10:28:43.414874 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.414849    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 10:28:43.415340 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.415304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zl4tt\""
Jul 29 10:28:43.416170 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.416139    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 10:28:43.416275 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.416210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 10:28:43.419784 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.419709    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r7d54"]
Jul 29 10:28:43.420725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.420702    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zbcld"]
Jul 29 10:28:43.501166 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:28:43.501351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-bound-sa-token\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501129    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:28:43.501351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-image-registry-private-configuration\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501305    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:28:43.501351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501316    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdb9f707-359d-4e5d-a6bc-6c3da935da90-ca-trust-extracted\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-certificates\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501388    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b8bb7ea-2050-4f1d-a5ed-1957c71ae978-cert\") pod \"ingress-canary-r7d54\" (UID: \"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978\") " pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l482g\" (UniqueName: \"kubernetes.io/projected/0b8bb7ea-2050-4f1d-a5ed-1957c71ae978-kube-api-access-l482g\") pod \"ingress-canary-r7d54\" (UID: \"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978\") " pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvhh2\" (UniqueName: \"kubernetes.io/projected/22601efc-2958-474f-8590-3725796c9e2c-kube-api-access-kvhh2\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-trusted-ca\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501488    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlw7w\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-kube-api-access-dlw7w\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501547    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-installation-pull-secrets\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22601efc-2958-474f-8590-3725796c9e2c-config-volume\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.501650 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22601efc-2958-474f-8590-3725796c9e2c-tmp-dir\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.502072 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.501699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.503976 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.503955    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 10:28:43.504103 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.503985    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 10:28:43.504103 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.503994    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 10:28:43.504201 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.504134    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l8nkz\""
Jul 29 10:28:43.504372 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.504353    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 10:28:43.504477 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.504425    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wmbjw\""
Jul 29 10:28:43.550268 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.550229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:43.603134 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-bound-sa-token\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603134 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603108    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-image-registry-private-configuration\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603366 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:43.603289    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:28:43.603366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdb9f707-359d-4e5d-a6bc-6c3da935da90-ca-trust-extracted\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-certificates\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b8bb7ea-2050-4f1d-a5ed-1957c71ae978-cert\") pod \"ingress-canary-r7d54\" (UID: \"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978\") " pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.603517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l482g\" (UniqueName: \"kubernetes.io/projected/0b8bb7ea-2050-4f1d-a5ed-1957c71ae978-kube-api-access-l482g\") pod \"ingress-canary-r7d54\" (UID: \"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978\") " pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.603517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvhh2\" (UniqueName: \"kubernetes.io/projected/22601efc-2958-474f-8590-3725796c9e2c-kube-api-access-kvhh2\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.603707 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:43.603335    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:28:43.603773 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:43.603753    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:44.103730757 +0000 UTC m=+37.092580198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:28:43.603917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603784    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-trusted-ca\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlw7w\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-kube-api-access-dlw7w\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-installation-pull-secrets\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603855    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdb9f707-359d-4e5d-a6bc-6c3da935da90-ca-trust-extracted\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.603917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22601efc-2958-474f-8590-3725796c9e2c-config-volume\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.603917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22601efc-2958-474f-8590-3725796c9e2c-tmp-dir\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.604219 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.603946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.604219 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:43.604053    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:28:43.604219 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:43.604090    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:28:44.104078597 +0000 UTC m=+37.092928044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:28:43.604219 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.604196    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-certificates\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.604492 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.604455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22601efc-2958-474f-8590-3725796c9e2c-tmp-dir\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.604716 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.604662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/22601efc-2958-474f-8590-3725796c9e2c-config-volume\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.604951 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.604892    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-trusted-ca\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.608150 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.608121    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-image-registry-private-configuration\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.608692 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.608672    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b8bb7ea-2050-4f1d-a5ed-1957c71ae978-cert\") pod \"ingress-canary-r7d54\" (UID: \"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978\") " pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.608860 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.608834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-installation-pull-secrets\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.612514 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.612488    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-bound-sa-token\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.612514 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.612502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvhh2\" (UniqueName: \"kubernetes.io/projected/22601efc-2958-474f-8590-3725796c9e2c-kube-api-access-kvhh2\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:43.612695 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.612520    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l482g\" (UniqueName: \"kubernetes.io/projected/0b8bb7ea-2050-4f1d-a5ed-1957c71ae978-kube-api-access-l482g\") pod \"ingress-canary-r7d54\" (UID: \"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978\") " pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:43.612695 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.612632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlw7w\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-kube-api-access-dlw7w\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:43.653055 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.653002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:43.730065 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:43.730033    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r7d54"
Jul 29 10:28:44.108093 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:44.108047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:44.108300 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:44.108132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:44.108300 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:44.108218    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:28:44.108300 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:44.108265    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:28:44.108300 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:44.108279    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:28:44.108538 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:44.108303    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:28:45.108280796 +0000 UTC m=+38.097130231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:28:44.108538 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:44.108343    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:45.10831479 +0000 UTC m=+38.097164225 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:28:44.768589 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:44.768263    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r7d54"]
Jul 29 10:28:44.773472 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:28:44.773443    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8bb7ea_2050_4f1d_a5ed_1957c71ae978.slice/crio-c02ecdf1f943394a5a756b0e45619b10c97d8daae5c1c9952d82ef783e08205e WatchSource:0}: Error finding container c02ecdf1f943394a5a756b0e45619b10c97d8daae5c1c9952d82ef783e08205e: Status 404 returned error can't find the container with id c02ecdf1f943394a5a756b0e45619b10c97d8daae5c1c9952d82ef783e08205e
Jul 29 10:28:45.116014 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:45.115974    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:45.116195 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:45.116051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:45.116195 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:45.116143    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:28:45.116270 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:45.116207    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:28:47.116190784 +0000 UTC m=+40.105040222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:28:45.116270 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:45.116152    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:28:45.116270 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:45.116250    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:28:45.116411 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:45.116287    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:47.116276177 +0000 UTC m=+40.105125610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:28:45.739888 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:45.739845    2562 generic.go:356] "Generic (PLEG): container finished" podID="abd0629b-e500-4831-b288-292c7a8c8c16" containerID="5577430ee882a0c0b9a16281bdcba345601e228d230bcddda5c2adfe0cbfa846" exitCode=0
Jul 29 10:28:45.740107 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:45.739930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerDied","Data":"5577430ee882a0c0b9a16281bdcba345601e228d230bcddda5c2adfe0cbfa846"}
Jul 29 10:28:45.741218 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:45.741106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r7d54" event={"ID":"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978","Type":"ContainerStarted","Data":"c02ecdf1f943394a5a756b0e45619b10c97d8daae5c1c9952d82ef783e08205e"}
Jul 29 10:28:46.747020 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:46.746980    2562 generic.go:356] "Generic (PLEG): container finished" podID="abd0629b-e500-4831-b288-292c7a8c8c16" containerID="b8e7711482c5856e4fc51b4abea008d4c17ac3f812eca3fb9e647cd4f083ff4f" exitCode=0
Jul 29 10:28:46.747488 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:46.747068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerDied","Data":"b8e7711482c5856e4fc51b4abea008d4c17ac3f812eca3fb9e647cd4f083ff4f"}
Jul 29 10:28:47.132592 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:47.132492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:47.132760 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:47.132672    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:28:47.132760 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:47.132696    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:28:47.132760 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:47.132717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:47.132760 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:47.132750    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:51.132732461 +0000 UTC m=+44.121581896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:28:47.132952 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:47.132803    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:28:47.132952 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:47.132873    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:28:51.132856692 +0000 UTC m=+44.121706128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:28:47.754701 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:47.754471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" event={"ID":"abd0629b-e500-4831-b288-292c7a8c8c16","Type":"ContainerStarted","Data":"72abd8474add568dedc9cc0c99e338d6fa87eb7641a65fd5b9800eaef9829728"}
Jul 29 10:28:47.779493 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:47.779433    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x8m2t" podStartSLOduration=9.651190889 podStartE2EDuration="39.779409031s" totalImagesPullingTime="30.128218142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:47.777770164 +0000 UTC m=+40.766619624" watchObservedRunningTime="2026-07-29 10:28:47.779409031 +0000 UTC m=+40.768258489"
Jul 29 10:28:47.903468 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:47.903432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:48.002413 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.002337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:48.316624 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.316542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:48.420067 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.420029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:48.758465 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.758407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r7d54" event={"ID":"0b8bb7ea-2050-4f1d-a5ed-1957c71ae978","Type":"ContainerStarted","Data":"5bd48d638a7269247037939f0033a2b2664b951a9a15dea8c550b40de0db535a"}
Jul 29 10:28:48.769786 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.769746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:48.776813 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.776758    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r7d54" podStartSLOduration=2.8465862250000002 podStartE2EDuration="5.776743354s" totalImagesPullingTime="2.930157129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:28:48.775145804 +0000 UTC m=+41.763995262" watchObservedRunningTime="2026-07-29 10:28:48.776743354 +0000 UTC m=+41.765592788"
Jul 29 10:28:48.871339 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.871258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:48.935123 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:48.935092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:49.689908 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:49.689870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:49.758017 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:49.757984    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:50.676816 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:50.676782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:50.745742 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:50.745703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:51.163925 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:51.163884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:51.164077 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:51.163944    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:51.164077 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:51.164031    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:28:51.164077 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:51.164050    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:28:51.164077 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:51.164051    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:28:51.164220 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:51.164104    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:28:59.164087887 +0000 UTC m=+52.152937321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:28:51.164220 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:51.164122    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:28:59.164113094 +0000 UTC m=+52.152962529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:28:58.598777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:58.598741    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:28:59.229608 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:59.229563    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:28:59.229788 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:28:59.229640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:28:59.229788 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:59.229731    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:28:59.229788 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:59.229747    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:28:59.229788 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:59.229751    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:28:59.229919 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:59.229805    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:29:15.229791357 +0000 UTC m=+68.218640792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:28:59.229919 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:28:59.229821    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:15.229815151 +0000 UTC m=+68.218664585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:29:00.707122 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:00.707085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:01.298869 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:01.298836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:01.399698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:01.399665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:01.637764 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:01.637675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:02.320104 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:02.320063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:02.583421 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:02.583315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:04.766288 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:04.766248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:04.867081 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:04.867050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:05.279899 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:05.279867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:05.336889 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:05.336856    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:06.784728 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.784689    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-sb872"]
Jul 29 10:29:06.786809 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.786793    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"]
Jul 29 10:29:06.786939 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.786926    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.788752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.788732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"
Jul 29 10:29:06.789709 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.789691    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 10:29:06.789892 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.789878    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pq9d7\""
Jul 29 10:29:06.789955 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.789881    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:06.789955 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.789888    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:06.791079 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.791058    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 10:29:06.792564 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.792403    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:06.792564 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.792494    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-4brcs\""
Jul 29 10:29:06.792711 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.792528    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:06.795187 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.795165    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-sb872"]
Jul 29 10:29:06.796107 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.796088    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 10:29:06.809944 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.809913    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"]
Jul 29 10:29:06.872504 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.872471    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"]
Jul 29 10:29:06.874371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.874355    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.877160 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.877123    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 10:29:06.877477 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.877462    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5jvjf\""
Jul 29 10:29:06.877553 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.877500    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:06.877601 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.877565    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 10:29:06.877601 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.877500    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:06.884454 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.884430    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"]
Jul 29 10:29:06.884872 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.884850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnngq\" (UniqueName: \"kubernetes.io/projected/0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb-kube-api-access-dnngq\") pod \"volume-data-source-validator-549c6d54cf-4td4l\" (UID: \"0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"
Jul 29 10:29:06.884943 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.884888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7ee7cb80-9446-46f9-ad0b-e21752077bea-config\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.884943 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.884920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59774\" (UniqueName: \"kubernetes.io/projected/7ee7cb80-9446-46f9-ad0b-e21752077bea-kube-api-access-59774\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.885014 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.884985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ee7cb80-9446-46f9-ad0b-e21752077bea-serving-cert\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.885046 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.885038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ee7cb80-9446-46f9-ad0b-e21752077bea-trusted-ca\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.885078 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.885063    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7ab95840-c1d9-48ef-a841-8a7bf578ec09-config\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.885111 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.885097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42lnd\" (UniqueName: \"kubernetes.io/projected/7ab95840-c1d9-48ef-a841-8a7bf578ec09-kube-api-access-42lnd\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.885158 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.885141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ab95840-c1d9-48ef-a841-8a7bf578ec09-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.982890 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.982849    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"]
Jul 29 10:29:06.984766 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.984745    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:06.985822 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.985788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2caed74b-6436-4709-9330-ede74f2cb0c2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:06.985822 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.985824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59774\" (UniqueName: \"kubernetes.io/projected/7ee7cb80-9446-46f9-ad0b-e21752077bea-kube-api-access-59774\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.985976 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.985894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ee7cb80-9446-46f9-ad0b-e21752077bea-serving-cert\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.985976 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.985932    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ee7cb80-9446-46f9-ad0b-e21752077bea-trusted-ca\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.985976 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.985952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7ab95840-c1d9-48ef-a841-8a7bf578ec09-config\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.986108 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.985991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42lnd\" (UniqueName: \"kubernetes.io/projected/7ab95840-c1d9-48ef-a841-8a7bf578ec09-kube-api-access-42lnd\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.986108 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986021    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ab95840-c1d9-48ef-a841-8a7bf578ec09-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.986108 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2caed74b-6436-4709-9330-ede74f2cb0c2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:06.986254 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnngq\" (UniqueName: \"kubernetes.io/projected/0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb-kube-api-access-dnngq\") pod \"volume-data-source-validator-549c6d54cf-4td4l\" (UID: \"0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"
Jul 29 10:29:06.986254 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986156    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7ee7cb80-9446-46f9-ad0b-e21752077bea-config\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.986254 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljncw\" (UniqueName: \"kubernetes.io/projected/2caed74b-6436-4709-9330-ede74f2cb0c2-kube-api-access-ljncw\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:06.986563 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986539    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7ab95840-c1d9-48ef-a841-8a7bf578ec09-config\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.986867 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7ee7cb80-9446-46f9-ad0b-e21752077bea-config\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.986929 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.986872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7ee7cb80-9446-46f9-ad0b-e21752077bea-trusted-ca\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:06.988400 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.988378    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 10:29:06.988510 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.988384    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:06.988510 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.988493    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 10:29:06.988643 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.988567    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 10:29:06.988713 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.988696    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9x5bz\""
Jul 29 10:29:06.990229 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.990204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ab95840-c1d9-48ef-a841-8a7bf578ec09-serving-cert\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:06.990372 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:06.990250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ee7cb80-9446-46f9-ad0b-e21752077bea-serving-cert\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:07.006468 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.006437    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"]
Jul 29 10:29:07.011204 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.011181    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42lnd\" (UniqueName: \"kubernetes.io/projected/7ab95840-c1d9-48ef-a841-8a7bf578ec09-kube-api-access-42lnd\") pod \"service-ca-operator-5d6c75f84f-bkxld\" (UID: \"7ab95840-c1d9-48ef-a841-8a7bf578ec09\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:07.011441 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.011418    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59774\" (UniqueName: \"kubernetes.io/projected/7ee7cb80-9446-46f9-ad0b-e21752077bea-kube-api-access-59774\") pod \"console-operator-584dcb6b54-sb872\" (UID: \"7ee7cb80-9446-46f9-ad0b-e21752077bea\") " pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:07.012214 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.012195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnngq\" (UniqueName: \"kubernetes.io/projected/0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb-kube-api-access-dnngq\") pod \"volume-data-source-validator-549c6d54cf-4td4l\" (UID: \"0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"
Jul 29 10:29:07.086932 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.086839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2caed74b-6436-4709-9330-ede74f2cb0c2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.086932 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.086902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljncw\" (UniqueName: \"kubernetes.io/projected/2caed74b-6436-4709-9330-ede74f2cb0c2-kube-api-access-ljncw\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.086932 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.086925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2caed74b-6436-4709-9330-ede74f2cb0c2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.087515 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.087494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2caed74b-6436-4709-9330-ede74f2cb0c2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.089206 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.089182    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2caed74b-6436-4709-9330-ede74f2cb0c2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.095953 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.095929    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljncw\" (UniqueName: \"kubernetes.io/projected/2caed74b-6436-4709-9330-ede74f2cb0c2-kube-api-access-ljncw\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss\" (UID: \"2caed74b-6436-4709-9330-ede74f2cb0c2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.100753 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.100731    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:07.105566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.105547    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"
Jul 29 10:29:07.134358 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.134299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:07.185269 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.184974    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"
Jul 29 10:29:07.235768 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.235732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:07.261163 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.261133    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-sb872"]
Jul 29 10:29:07.265055 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:07.265027    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee7cb80_9446_46f9_ad0b_e21752077bea.slice/crio-00fa4d0a40441a477f2448ec4ad0c640970758bfd429ab8527d9d8ab3671629a WatchSource:0}: Error finding container 00fa4d0a40441a477f2448ec4ad0c640970758bfd429ab8527d9d8ab3671629a: Status 404 returned error can't find the container with id 00fa4d0a40441a477f2448ec4ad0c640970758bfd429ab8527d9d8ab3671629a
Jul 29 10:29:07.275280 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.275253    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l"]
Jul 29 10:29:07.279563 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:07.279530    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba8dc12_e5c0_4be8_bd18_54dcbdfd28bb.slice/crio-7b3146532bf4f30cd686c43471cbcc308b076bc13f1817261345c49c93cb6a89 WatchSource:0}: Error finding container 7b3146532bf4f30cd686c43471cbcc308b076bc13f1817261345c49c93cb6a89: Status 404 returned error can't find the container with id 7b3146532bf4f30cd686c43471cbcc308b076bc13f1817261345c49c93cb6a89
Jul 29 10:29:07.298635 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.298604    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"
Jul 29 10:29:07.325262 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.325231    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld"]
Jul 29 10:29:07.328646 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:07.328601    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab95840_c1d9_48ef_a841_8a7bf578ec09.slice/crio-6607b65ae97a00dd3bf30498e7aaf3d93a85ec0d0d0c7832b94d703809158ef4 WatchSource:0}: Error finding container 6607b65ae97a00dd3bf30498e7aaf3d93a85ec0d0d0c7832b94d703809158ef4: Status 404 returned error can't find the container with id 6607b65ae97a00dd3bf30498e7aaf3d93a85ec0d0d0c7832b94d703809158ef4
Jul 29 10:29:07.426574 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.426537    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss"]
Jul 29 10:29:07.430310 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:07.430286    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2caed74b_6436_4709_9330_ede74f2cb0c2.slice/crio-1ead2eb55db16ec9772ec7ae0cee2578f018f103fb12aff55ff103929d1f5bb1 WatchSource:0}: Error finding container 1ead2eb55db16ec9772ec7ae0cee2578f018f103fb12aff55ff103929d1f5bb1: Status 404 returned error can't find the container with id 1ead2eb55db16ec9772ec7ae0cee2578f018f103fb12aff55ff103929d1f5bb1
Jul 29 10:29:07.805802 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.805756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l" event={"ID":"0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb","Type":"ContainerStarted","Data":"7b3146532bf4f30cd686c43471cbcc308b076bc13f1817261345c49c93cb6a89"}
Jul 29 10:29:07.807037 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.807012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" event={"ID":"7ee7cb80-9446-46f9-ad0b-e21752077bea","Type":"ContainerStarted","Data":"00fa4d0a40441a477f2448ec4ad0c640970758bfd429ab8527d9d8ab3671629a"}
Jul 29 10:29:07.808265 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.808243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld" event={"ID":"7ab95840-c1d9-48ef-a841-8a7bf578ec09","Type":"ContainerStarted","Data":"6607b65ae97a00dd3bf30498e7aaf3d93a85ec0d0d0c7832b94d703809158ef4"}
Jul 29 10:29:07.809378 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:07.809354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss" event={"ID":"2caed74b-6436-4709-9330-ede74f2cb0c2","Type":"ContainerStarted","Data":"1ead2eb55db16ec9772ec7ae0cee2578f018f103fb12aff55ff103929d1f5bb1"}
Jul 29 10:29:08.814548 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:08.814501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l" event={"ID":"0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb","Type":"ContainerStarted","Data":"bfff3b49bcde3e55f745ffcbcddd362e7d469e516e085f50f8d0335ef89c1f41"}
Jul 29 10:29:08.832436 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:08.832367    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-4td4l" podStartSLOduration=1.690801942 podStartE2EDuration="2.832343301s" totalImagesPullingTime="1.141541359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:08.831680322 +0000 UTC m=+61.820529779" watchObservedRunningTime="2026-07-29 10:29:08.832343301 +0000 UTC m=+61.821192757"
Jul 29 10:29:08.956422 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:08.956369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:09.067013 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:09.066885    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:09.733743 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:09.733712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4gwff"
Jul 29 10:29:09.893782 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:09.893747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:10.447543 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:10.447507    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:11.732354 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.732296    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-86nmd"]
Jul 29 10:29:11.735505 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.735484    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.746337 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.746294    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 10:29:11.746337 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.746294    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 10:29:11.747439 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.747420    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 10:29:11.747509 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.747480    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 10:29:11.749883 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.749862    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5579t\""
Jul 29 10:29:11.760509 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.760466    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-86nmd"]
Jul 29 10:29:11.773076 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.773036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:11.823742 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.823715    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/0.log"
Jul 29 10:29:11.823934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.823755    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ee7cb80-9446-46f9-ad0b-e21752077bea" containerID="d51410ca863d4e37e57d2b1cae2faafa48fc1efb6dc5211c17ea8f34373111d7" exitCode=255
Jul 29 10:29:11.823934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.823789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" event={"ID":"7ee7cb80-9446-46f9-ad0b-e21752077bea","Type":"ContainerDied","Data":"d51410ca863d4e37e57d2b1cae2faafa48fc1efb6dc5211c17ea8f34373111d7"}
Jul 29 10:29:11.824080 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.824064    2562 scope.go:117] "RemoveContainer" containerID="d51410ca863d4e37e57d2b1cae2faafa48fc1efb6dc5211c17ea8f34373111d7"
Jul 29 10:29:11.825484 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.825445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld" event={"ID":"7ab95840-c1d9-48ef-a841-8a7bf578ec09","Type":"ContainerStarted","Data":"d8d903f63db2d9d678f6ef7050a46b10da9be5c752881ce5279d61948a0dea82"}
Jul 29 10:29:11.826927 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.826898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss" event={"ID":"2caed74b-6436-4709-9330-ede74f2cb0c2","Type":"ContainerStarted","Data":"e5df9831d4239708f5f15221049b66246833f26f3e2489ecbad4c3cf44fbf593"}
Jul 29 10:29:11.827048 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.826984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-data-volume\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.827155 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.827134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75xq8\" (UniqueName: \"kubernetes.io/projected/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-api-access-75xq8\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.827211 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.827176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.827264 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.827229    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.827317 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.827294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-crio-socket\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.856818 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.856784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:11.876491 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.876444    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss" podStartSLOduration=2.486249238 podStartE2EDuration="5.876426239s" totalImagesPullingTime="3.390177001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:11.875386019 +0000 UTC m=+64.864235474" watchObservedRunningTime="2026-07-29 10:29:11.876426239 +0000 UTC m=+64.865275716"
Jul 29 10:29:11.908057 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.907012    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld" podStartSLOduration=2.412715777 podStartE2EDuration="5.906993788s" totalImagesPullingTime="3.494278011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:11.905963138 +0000 UTC m=+64.894812595" watchObservedRunningTime="2026-07-29 10:29:11.906993788 +0000 UTC m=+64.895843246"
Jul 29 10:29:11.928535 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.928496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75xq8\" (UniqueName: \"kubernetes.io/projected/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-api-access-75xq8\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.928689 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.928552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.928689 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.928607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.928689 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.928661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-crio-socket\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.928825 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.928763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-data-volume\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.929053 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.928996    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-crio-socket\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.929273 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:11.929249    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:11.929363 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:11.929249    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:29:11.929363 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:11.929347    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:12.42930219 +0000 UTC m=+65.418151628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:11.929490 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:11.929396    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:12.429372538 +0000 UTC m=+65.418221987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : configmap "kube-rbac-proxy" not found
Jul 29 10:29:11.929878 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.929857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-data-volume\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.938827 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.938804    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75xq8\" (UniqueName: \"kubernetes.io/projected/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-api-access-75xq8\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:11.993641 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:11.993552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:12.262790 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.262693    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"]
Jul 29 10:29:12.266096 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.266068    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"
Jul 29 10:29:12.269371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.269343    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 29 10:29:12.269505 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.269389    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 29 10:29:12.270147 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.270129    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-szg2t\""
Jul 29 10:29:12.278666 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.278636    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"]
Jul 29 10:29:12.284390 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.284358    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:12.331925 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.331882    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clcpz\" (UniqueName: \"kubernetes.io/projected/71b77379-e25b-45ae-af06-10b5868bb9ab-kube-api-access-clcpz\") pod \"migrator-6865c9967-8725r\" (UID: \"71b77379-e25b-45ae-af06-10b5868bb9ab\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"
Jul 29 10:29:12.373795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.373760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:12.433116 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.433073    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clcpz\" (UniqueName: \"kubernetes.io/projected/71b77379-e25b-45ae-af06-10b5868bb9ab-kube-api-access-clcpz\") pod \"migrator-6865c9967-8725r\" (UID: \"71b77379-e25b-45ae-af06-10b5868bb9ab\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"
Jul 29 10:29:12.433116 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.433119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:12.433348 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.433154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:12.433348 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:12.433223    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:29:12.433348 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:12.433277    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:12.433348 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:12.433297    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:13.433279153 +0000 UTC m=+66.422128591 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : configmap "kube-rbac-proxy" not found
Jul 29 10:29:12.433348 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:12.433347    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:13.433309315 +0000 UTC m=+66.422158759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:12.442679 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.442654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clcpz\" (UniqueName: \"kubernetes.io/projected/71b77379-e25b-45ae-af06-10b5868bb9ab-kube-api-access-clcpz\") pod \"migrator-6865c9967-8725r\" (UID: \"71b77379-e25b-45ae-af06-10b5868bb9ab\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"
Jul 29 10:29:12.474593 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.474559    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:12.575425 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.575342    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"
Jul 29 10:29:12.658144 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.658087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:12.704815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.704789    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-8725r"]
Jul 29 10:29:12.706866 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:12.706837    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b77379_e25b_45ae_af06_10b5868bb9ab.slice/crio-6919a63518bcba9af1936053d7649a048a70fe511b5d817852990d85abdaf082 WatchSource:0}: Error finding container 6919a63518bcba9af1936053d7649a048a70fe511b5d817852990d85abdaf082: Status 404 returned error can't find the container with id 6919a63518bcba9af1936053d7649a048a70fe511b5d817852990d85abdaf082
Jul 29 10:29:12.733721 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.733685    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:12.831571 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.831477    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r" event={"ID":"71b77379-e25b-45ae-af06-10b5868bb9ab","Type":"ContainerStarted","Data":"6919a63518bcba9af1936053d7649a048a70fe511b5d817852990d85abdaf082"}
Jul 29 10:29:12.832973 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.832952    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:29:12.833364 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.833348    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/0.log"
Jul 29 10:29:12.833459 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.833386    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ee7cb80-9446-46f9-ad0b-e21752077bea" containerID="7c51a9790f4e5d918fd970ebf25a6a6f39d8b4f3a2d9af4013471f7a3ea7dd3f" exitCode=255
Jul 29 10:29:12.833504 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.833474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" event={"ID":"7ee7cb80-9446-46f9-ad0b-e21752077bea","Type":"ContainerDied","Data":"7c51a9790f4e5d918fd970ebf25a6a6f39d8b4f3a2d9af4013471f7a3ea7dd3f"}
Jul 29 10:29:12.833541 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.833527    2562 scope.go:117] "RemoveContainer" containerID="d51410ca863d4e37e57d2b1cae2faafa48fc1efb6dc5211c17ea8f34373111d7"
Jul 29 10:29:12.833824 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:12.833808    2562 scope.go:117] "RemoveContainer" containerID="7c51a9790f4e5d918fd970ebf25a6a6f39d8b4f3a2d9af4013471f7a3ea7dd3f"
Jul 29 10:29:12.834011 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:12.833988    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-sb872_openshift-console-operator(7ee7cb80-9446-46f9-ad0b-e21752077bea)\"" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" podUID="7ee7cb80-9446-46f9-ad0b-e21752077bea"
Jul 29 10:29:13.240983 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.240937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:29:13.241183 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.241146    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:29:13.244256 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.244227    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 10:29:13.244256 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.244246    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 10:29:13.254615 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.254585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/198702a2-64b9-4023-84f4-cb0388b57f62-original-pull-secret\") pod \"global-pull-secret-syncer-xp6pm\" (UID: \"198702a2-64b9-4023-84f4-cb0388b57f62\") " pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:29:13.255068 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.255046    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 10:29:13.265582 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.265554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl9dl\" (UniqueName: \"kubernetes.io/projected/a2c68518-a780-4462-947d-f73790788f23-kube-api-access-hl9dl\") pod \"network-check-target-mj9l6\" (UID: \"a2c68518-a780-4462-947d-f73790788f23\") " pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:29:13.342309 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.342270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:29:13.347674 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.347639    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 10:29:13.352658 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.352625    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 10:29:13.352797 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.352713    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs podName:79f1afb4-3c15-4858-8291-48c7603f443a nodeName:}" failed. No retries permitted until 2026-07-29 10:30:17.352689436 +0000 UTC m=+130.341538871 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs") pod "network-metrics-daemon-hrntc" (UID: "79f1afb4-3c15-4858-8291-48c7603f443a") : secret "metrics-daemon-secret" not found
Jul 29 10:29:13.443262 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.443215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:13.443262 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.443261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:13.443521 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.443362    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:29:13.443521 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.443400    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:13.443521 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.443447    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:15.443428315 +0000 UTC m=+68.432277748 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : configmap "kube-rbac-proxy" not found
Jul 29 10:29:13.443521 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.443462    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:15.443455898 +0000 UTC m=+68.432305332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:13.485482 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.485447    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vkshd_c329c2e8-06df-4cad-b6ea-5cff76ef5978/dns-node-resolver/0.log"
Jul 29 10:29:13.520218 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.520130    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xp6pm"
Jul 29 10:29:13.529187 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.529162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wmbjw\""
Jul 29 10:29:13.536934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.536906    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:29:13.665039 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.665004    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xp6pm"]
Jul 29 10:29:13.682433 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.682397    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mj9l6"]
Jul 29 10:29:13.777658 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.777570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:13.838385 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.838352    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:29:13.838803 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.838781    2562 scope.go:117] "RemoveContainer" containerID="7c51a9790f4e5d918fd970ebf25a6a6f39d8b4f3a2d9af4013471f7a3ea7dd3f"
Jul 29 10:29:13.838988 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:13.838961    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-sb872_openshift-console-operator(7ee7cb80-9446-46f9-ad0b-e21752077bea)\"" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" podUID="7ee7cb80-9446-46f9-ad0b-e21752077bea"
Jul 29 10:29:13.844770 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:13.844725    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:13.866928 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:13.866884    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod198702a2_64b9_4023_84f4_cb0388b57f62.slice/crio-a828e5232a6a571a9014aebd7d02f5c6f836a89c5328162954dd162ec8a7a4e5 WatchSource:0}: Error finding container a828e5232a6a571a9014aebd7d02f5c6f836a89c5328162954dd162ec8a7a4e5: Status 404 returned error can't find the container with id a828e5232a6a571a9014aebd7d02f5c6f836a89c5328162954dd162ec8a7a4e5
Jul 29 10:29:13.867679 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:13.867428    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c68518_a780_4462_947d_f73790788f23.slice/crio-15b21a7ba4754ea71365afb6674ce4a77a269d6cdea5b5c5ed8835d5f8888400 WatchSource:0}: Error finding container 15b21a7ba4754ea71365afb6674ce4a77a269d6cdea5b5c5ed8835d5f8888400: Status 404 returned error can't find the container with id 15b21a7ba4754ea71365afb6674ce4a77a269d6cdea5b5c5ed8835d5f8888400
Jul 29 10:29:14.272055 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.272010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:14.284830 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.284793    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sx4n7_0a034c05-e7ae-4fd3-b69b-766314822683/node-ca/0.log"
Jul 29 10:29:14.338720 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.338679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:14.845173 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.845133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r" event={"ID":"71b77379-e25b-45ae-af06-10b5868bb9ab","Type":"ContainerStarted","Data":"6efd3e81610ba85e000a2180fa464a9efd46c12ed490e8c35f0614a84aba2787"}
Jul 29 10:29:14.845738 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.845196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r" event={"ID":"71b77379-e25b-45ae-af06-10b5868bb9ab","Type":"ContainerStarted","Data":"72de80b24fe4fbe914dc6e087f4a6febf5f48091be0563d2714ac6a46beaf971"}
Jul 29 10:29:14.846902 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.846845    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mj9l6" event={"ID":"a2c68518-a780-4462-947d-f73790788f23","Type":"ContainerStarted","Data":"15b21a7ba4754ea71365afb6674ce4a77a269d6cdea5b5c5ed8835d5f8888400"}
Jul 29 10:29:14.848294 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.848266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xp6pm" event={"ID":"198702a2-64b9-4023-84f4-cb0388b57f62","Type":"ContainerStarted","Data":"a828e5232a6a571a9014aebd7d02f5c6f836a89c5328162954dd162ec8a7a4e5"}
Jul 29 10:29:14.866988 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:14.866917    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-8725r" podStartSLOduration=1.664058667 podStartE2EDuration="2.866892346s" totalImagesPullingTime="1.202833679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:14.865702487 +0000 UTC m=+67.854551944" watchObservedRunningTime="2026-07-29 10:29:14.866892346 +0000 UTC m=+67.855741805"
Jul 29 10:29:15.085814 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:15.085782    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r7d54_0b8bb7ea-2050-4f1d-a5ed-1957c71ae978/serve-healthcheck-canary/0.log"
Jul 29 10:29:15.128399 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:15.128282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:15.261895 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:15.261846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:29:15.262086 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:15.261937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:29:15.262086 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.262026    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 10:29:15.262086 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.262049    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c48fd7547-8z9mq: secret "image-registry-tls" not found
Jul 29 10:29:15.262086 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.262084    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 10:29:15.262268 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.262119    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls podName:bdb9f707-359d-4e5d-a6bc-6c3da935da90 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:47.262099385 +0000 UTC m=+100.250948843 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls") pod "image-registry-6c48fd7547-8z9mq" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90") : secret "image-registry-tls" not found
Jul 29 10:29:15.262268 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.262138    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls podName:22601efc-2958-474f-8590-3725796c9e2c nodeName:}" failed. No retries permitted until 2026-07-29 10:29:47.262131249 +0000 UTC m=+100.250980682 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls") pod "dns-default-zbcld" (UID: "22601efc-2958-474f-8590-3725796c9e2c") : secret "dns-default-metrics-tls" not found
Jul 29 10:29:15.463608 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:15.463518    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:15.463608 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:15.463581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:15.463896 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.463659    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:29:15.463896 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.463743    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:15.463896 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.463746    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:19.463724677 +0000 UTC m=+72.452574122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : configmap "kube-rbac-proxy" not found
Jul 29 10:29:15.463896 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:15.463812    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:19.463795575 +0000 UTC m=+72.452645022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:17.101064 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:17.101020    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:17.101576 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:17.101129    2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:17.101643 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:17.101610    2562 scope.go:117] "RemoveContainer" containerID="7c51a9790f4e5d918fd970ebf25a6a6f39d8b4f3a2d9af4013471f7a3ea7dd3f"
Jul 29 10:29:17.101828 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:17.101803    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-sb872_openshift-console-operator(7ee7cb80-9446-46f9-ad0b-e21752077bea)\"" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" podUID="7ee7cb80-9446-46f9-ad0b-e21752077bea"
Jul 29 10:29:18.146486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.146445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:18.216444 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.216395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:18.864123 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.864082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mj9l6" event={"ID":"a2c68518-a780-4462-947d-f73790788f23","Type":"ContainerStarted","Data":"7d37f4c59cc347b08159fcdd9298d4939e6c3a91b30c4f04ccfb26d43afd71dc"}
Jul 29 10:29:18.864340 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.864192    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:29:18.865670 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.865642    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xp6pm" event={"ID":"198702a2-64b9-4023-84f4-cb0388b57f62","Type":"ContainerStarted","Data":"57c2258a1136f8935555651675bddf54eee1faf5d177d032054346957d957a0f"}
Jul 29 10:29:18.882790 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.882694    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mj9l6" podStartSLOduration=66.110770399 podStartE2EDuration="1m10.882677546s" totalImagesPullingTime="4.771907147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:18.881766501 +0000 UTC m=+71.870615947" watchObservedRunningTime="2026-07-29 10:29:18.882677546 +0000 UTC m=+71.871527002"
Jul 29 10:29:18.900654 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:18.900599    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xp6pm" podStartSLOduration=66.11525665 podStartE2EDuration="1m10.900581657s" totalImagesPullingTime="4.785325007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:18.899858879 +0000 UTC m=+71.888708336" watchObservedRunningTime="2026-07-29 10:29:18.900581657 +0000 UTC m=+71.889431112"
Jul 29 10:29:19.251694 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.251657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:19.352294 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.352253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:19.498084 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.498044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:19.498293 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.498091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:19.498293 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:19.498186    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 10:29:19.498293 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:19.498258    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:27.498242209 +0000 UTC m=+80.487091642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : configmap "kube-rbac-proxy" not found
Jul 29 10:29:19.498293 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:19.498262    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:19.498497 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:19.498314    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:27.498298808 +0000 UTC m=+80.487148243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:19.517344 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.517223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:19.618435 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.618398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:19.754363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.754315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:19.854808 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.854707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:19.955409 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:19.955351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:20.550259 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:20.550224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:21.153170 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:21.153132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:21.560509 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:21.560474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:21.743779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:21.743732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:22.344937 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:22.344891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:22.583482 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:22.583446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:22.944368 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:22.944314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:23.005754 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:23.005719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:23.346301 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:23.346265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:23.550035 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:23.549995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:24.156179 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:24.156140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:24.257223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:24.257190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:24.360009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:24.359962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:24.834144 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:24.834088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:24.935880 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:24.935839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:24.997711 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:24.997673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:25.347101 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:25.347068    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:25.961982 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:25.961922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:26.529510 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:26.529470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:26.770251 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:26.770219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:26.831206 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:26.831097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:26.947299 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:26.947260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:27.342827 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:27.342793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:27.565458 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:27.565403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:27.565458 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:27.565461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:27.565895 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:27.565545    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:27.565895 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:27.565608    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls podName:c44085eb-1b3d-4591-9baf-d9f38d5f47a2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:43.565593893 +0000 UTC m=+96.554443327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-86nmd" (UID: "c44085eb-1b3d-4591-9baf-d9f38d5f47a2") : secret "insights-runtime-extractor-tls" not found
Jul 29 10:29:27.565996 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:27.565974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:27.985147 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:27.985114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:28.162591 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:28.162553    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:28.758915 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:28.758874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:29.143765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.143680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:29.501561 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.501529    2562 scope.go:117] "RemoveContainer" containerID="7c51a9790f4e5d918fd970ebf25a6a6f39d8b4f3a2d9af4013471f7a3ea7dd3f"
Jul 29 10:29:29.557302 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.557263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:29.768832 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.768727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:29.898649 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.898622    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:29:29.898813 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.898688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" event={"ID":"7ee7cb80-9446-46f9-ad0b-e21752077bea","Type":"ContainerStarted","Data":"f871a10006176d7bd6befc3beff8fcaf41a54db521d03530370c0c95bdec23cb"}
Jul 29 10:29:29.898978 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.898960    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:29.924762 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.924696    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-sb872" podStartSLOduration=20.363874824 podStartE2EDuration="23.924675963s" totalImagesPullingTime="3.560801139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:29.923292362 +0000 UTC m=+82.912141818" watchObservedRunningTime="2026-07-29 10:29:29.924675963 +0000 UTC m=+82.913525420"
Jul 29 10:29:29.956337 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:29.956285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:30.100472 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.100389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-sb872"
Jul 29 10:29:30.311309 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.311258    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:30.407206 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.407118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:30.476092 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.476053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:30.591842 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.591803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:30.782214 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.782178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:30.971658 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:30.971616    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:31.086451 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:31.086353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:31.191797 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:31.191756    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:32.215882 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:32.215847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:32.394418 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:32.394379    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:32.692371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:32.692336    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:33.009587 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:33.009554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:35.663816 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:35.663782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:35.732180 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:35.732150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:36.645429 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.645396    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-hjpt2"]
Jul 29 10:29:36.649921 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.649897    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:36.653940 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.653914    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 10:29:36.654526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.654504    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 10:29:36.654637 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.654530    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dn2cd\""
Jul 29 10:29:36.660772 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.660747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:36.668842 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.668813    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-hjpt2"]
Jul 29 10:29:36.736398 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.736359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbdmz\" (UniqueName: \"kubernetes.io/projected/f68b5087-14ea-4fa6-9ff6-3b5870681402-kube-api-access-sbdmz\") pod \"downloads-7c97dc4555-hjpt2\" (UID: \"f68b5087-14ea-4fa6-9ff6-3b5870681402\") " pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:36.750446 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.750416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:36.837631 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.837596    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbdmz\" (UniqueName: \"kubernetes.io/projected/f68b5087-14ea-4fa6-9ff6-3b5870681402-kube-api-access-sbdmz\") pod \"downloads-7c97dc4555-hjpt2\" (UID: \"f68b5087-14ea-4fa6-9ff6-3b5870681402\") " pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:36.846660 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.846624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbdmz\" (UniqueName: \"kubernetes.io/projected/f68b5087-14ea-4fa6-9ff6-3b5870681402-kube-api-access-sbdmz\") pod \"downloads-7c97dc4555-hjpt2\" (UID: \"f68b5087-14ea-4fa6-9ff6-3b5870681402\") " pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:36.849068 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.849023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:36.948720 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.948629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:36.959196 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:36.959155    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:37.065973 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:37.065936    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:37.095334 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:37.095292    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-hjpt2"]
Jul 29 10:29:37.098598 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:37.098568    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf68b5087_14ea_4fa6_9ff6_3b5870681402.slice/crio-b7849003c320ab3383fc89d42cff652d9db7a542752fefa7235b35addee2f927 WatchSource:0}: Error finding container b7849003c320ab3383fc89d42cff652d9db7a542752fefa7235b35addee2f927: Status 404 returned error can't find the container with id b7849003c320ab3383fc89d42cff652d9db7a542752fefa7235b35addee2f927
Jul 29 10:29:37.929143 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:37.929100    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-hjpt2" event={"ID":"f68b5087-14ea-4fa6-9ff6-3b5870681402","Type":"ContainerStarted","Data":"b7849003c320ab3383fc89d42cff652d9db7a542752fefa7235b35addee2f927"}
Jul 29 10:29:39.706619 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:39.706578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:39.808782 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:39.808724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:41.903133 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:41.903099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:42.004172 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:42.004134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:42.510672 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:42.510639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:42.610701 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:42.610645    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:42.697600 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:42.697567    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:42.907144 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:42.907058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:43.595826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:43.595787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:43.598177 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:43.598147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c44085eb-1b3d-4591-9baf-d9f38d5f47a2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-86nmd\" (UID: \"c44085eb-1b3d-4591-9baf-d9f38d5f47a2\") " pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:43.846039 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:43.845950    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-86nmd"
Jul 29 10:29:43.996357 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:43.996309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-86nmd"]
Jul 29 10:29:44.000265 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:44.000237    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44085eb_1b3d_4591_9baf_d9f38d5f47a2.slice/crio-ecbe63edcb31d8f1ef86aaf04919cdf84e61fe3e91ac172cb9c58b94dd77b8bd WatchSource:0}: Error finding container ecbe63edcb31d8f1ef86aaf04919cdf84e61fe3e91ac172cb9c58b94dd77b8bd: Status 404 returned error can't find the container with id ecbe63edcb31d8f1ef86aaf04919cdf84e61fe3e91ac172cb9c58b94dd77b8bd
Jul 29 10:29:44.962175 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:44.962134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86nmd" event={"ID":"c44085eb-1b3d-4591-9baf-d9f38d5f47a2","Type":"ContainerStarted","Data":"ecbe63edcb31d8f1ef86aaf04919cdf84e61fe3e91ac172cb9c58b94dd77b8bd"}
Jul 29 10:29:45.709491 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:45.709452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:45.789577 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:45.789466    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:45.872270 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:45.872219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:47.325248 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.325201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:29:47.325750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.325289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:29:47.327891 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.327864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/22601efc-2958-474f-8590-3725796c9e2c-metrics-tls\") pod \"dns-default-zbcld\" (UID: \"22601efc-2958-474f-8590-3725796c9e2c\") " pod="openshift-dns/dns-default-zbcld"
Jul 29 10:29:47.328445 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.328413    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"image-registry-6c48fd7547-8z9mq\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") " pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:29:47.606854 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.606768    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6wkgt\""
Jul 29 10:29:47.613803 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.613773    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:29:47.626151 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.626120    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rxxsv\""
Jul 29 10:29:47.633834 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.633803    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zbcld"
Jul 29 10:29:47.833627 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.833588    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:47.967753 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:47.967586    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:48.414886 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.414853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:48.499662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.499628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:48.814416 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.814383    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd78686d-ths5q"]
Jul 29 10:29:48.837410 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.837372    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd78686d-ths5q"]
Jul 29 10:29:48.837595 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.837567    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.841969 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.841947    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 10:29:48.842139 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.842068    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 10:29:48.843400 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.843378    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 10:29:48.843520 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.843404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 10:29:48.843520 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.843404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 10:29:48.843795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.843773    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8lgqv\""
Jul 29 10:29:48.868098 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.868062    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:48.938292 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.938259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-oauth-serving-cert\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.938487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.938410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-oauth-config\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.938487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.938458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-serving-cert\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.938487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.938482    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhl2h\" (UniqueName: \"kubernetes.io/projected/1d83814a-604e-43ba-ae35-538cf7dce61c-kube-api-access-jhl2h\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.938620 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.938513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-service-ca\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.938620 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.938559    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-console-config\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:48.963622 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:48.963582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:49.037779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.037734    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:49.039732 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.039708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-console-config\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.039871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.039743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-oauth-serving-cert\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.039871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.039800    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-oauth-config\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.039871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.039824    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-serving-cert\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.039871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.039849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhl2h\" (UniqueName: \"kubernetes.io/projected/1d83814a-604e-43ba-ae35-538cf7dce61c-kube-api-access-jhl2h\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.040074 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.039925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-service-ca\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.040594 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.040548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-console-config\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.040594 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.040577    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-oauth-serving-cert\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.040740 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.040662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-service-ca\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.042697 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.042669    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-serving-cert\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.042942 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.042926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-oauth-config\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.049456 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.049430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhl2h\" (UniqueName: \"kubernetes.io/projected/1d83814a-604e-43ba-ae35-538cf7dce61c-kube-api-access-jhl2h\") pod \"console-cd78686d-ths5q\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") " pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.149743 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.149649    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:29:49.783660 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.783612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:49.871908 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:49.871875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mj9l6"
Jul 29 10:29:50.209564 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:50.209426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:50.277560 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:50.277526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:50.801317 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:50.801279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:50.903299 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:50.903260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:51.809933 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:51.809898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:51.898183 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:51.898145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:51.999979 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:51.999944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:54.005395 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:54.005346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:54.105809 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:54.105774    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:56.578508 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.577962    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd78686d-ths5q"]
Jul 29 10:29:56.584216 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:56.584173    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d83814a_604e_43ba_ae35_538cf7dce61c.slice/crio-b49ca240fc7a3fb3a08f4b807babdc6e94c971f2e62ad6db1613749c000fa023 WatchSource:0}: Error finding container b49ca240fc7a3fb3a08f4b807babdc6e94c971f2e62ad6db1613749c000fa023: Status 404 returned error can't find the container with id b49ca240fc7a3fb3a08f4b807babdc6e94c971f2e62ad6db1613749c000fa023
Jul 29 10:29:56.598820 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.598783    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zbcld"]
Jul 29 10:29:56.602846 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:56.602818    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22601efc_2958_474f_8590_3725796c9e2c.slice/crio-23e8fcaebd8f278448ac4558bc1dd2e2311c30945e98b4fd32a4e960b224cea8 WatchSource:0}: Error finding container 23e8fcaebd8f278448ac4558bc1dd2e2311c30945e98b4fd32a4e960b224cea8: Status 404 returned error can't find the container with id 23e8fcaebd8f278448ac4558bc1dd2e2311c30945e98b4fd32a4e960b224cea8
Jul 29 10:29:56.620043 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.619993    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c48fd7547-8z9mq"]
Jul 29 10:29:56.623554 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:56.623517    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb9f707_359d_4e5d_a6bc_6c3da935da90.slice/crio-64fa466872beb710eab657ee8cd7e5e75676d3253099781e526f85cd7710fcad WatchSource:0}: Error finding container 64fa466872beb710eab657ee8cd7e5e75676d3253099781e526f85cd7710fcad: Status 404 returned error can't find the container with id 64fa466872beb710eab657ee8cd7e5e75676d3253099781e526f85cd7710fcad
Jul 29 10:29:56.824516 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.824462    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:56.899340 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.898624    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"]
Jul 29 10:29:56.903564 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.903530    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:56.907268 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.907016    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 10:29:56.907268 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.907127    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-r52k2\""
Jul 29 10:29:56.908599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.908576    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 10:29:56.908729 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.908582    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 10:29:56.908729 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.908715    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 10:29:56.908847 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.908787    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 10:29:56.908847 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.908797    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 10:29:56.921643 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.921603    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"]
Jul 29 10:29:56.925886 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.925853    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hk5zv"]
Jul 29 10:29:56.933696 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.930894    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:56.937599 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.937564    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 10:29:56.937758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.937620    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dcmhg\""
Jul 29 10:29:56.937758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.937620    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 10:29:56.941360 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.940610    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 10:29:56.947818 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:56.947487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.005725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.005355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.005725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.005407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.005725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.005461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.005725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.005488    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4plqf\" (UniqueName: \"kubernetes.io/projected/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-api-access-4plqf\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.005725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.005549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.005725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.005578    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.011656 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.011246    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-hjpt2" event={"ID":"f68b5087-14ea-4fa6-9ff6-3b5870681402","Type":"ContainerStarted","Data":"335d58d089cc64902006ad95837d7336f875fae16b9005007e65dd7b5ed91a55"}
Jul 29 10:29:57.011656 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.011590    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:57.014687 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.014630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86nmd" event={"ID":"c44085eb-1b3d-4591-9baf-d9f38d5f47a2","Type":"ContainerStarted","Data":"6eb67cc408a9d58f507b471ea8768aef39808bc06a76c86add6036ffce6bddff"}
Jul 29 10:29:57.018360 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.017654    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-hjpt2"
Jul 29 10:29:57.018360 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.017893    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd78686d-ths5q" event={"ID":"1d83814a-604e-43ba-ae35-538cf7dce61c","Type":"ContainerStarted","Data":"b49ca240fc7a3fb3a08f4b807babdc6e94c971f2e62ad6db1613749c000fa023"}
Jul 29 10:29:57.021916 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.020401    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zbcld" event={"ID":"22601efc-2958-474f-8590-3725796c9e2c","Type":"ContainerStarted","Data":"23e8fcaebd8f278448ac4558bc1dd2e2311c30945e98b4fd32a4e960b224cea8"}
Jul 29 10:29:57.024462 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.023708    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq" event={"ID":"bdb9f707-359d-4e5d-a6bc-6c3da935da90","Type":"ContainerStarted","Data":"73cbf0034a763d52f816b187b972423ee758c3932e230e1a1b6725d9e666d89e"}
Jul 29 10:29:57.024462 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.023760    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq" event={"ID":"bdb9f707-359d-4e5d-a6bc-6c3da935da90","Type":"ContainerStarted","Data":"64fa466872beb710eab657ee8cd7e5e75676d3253099781e526f85cd7710fcad"}
Jul 29 10:29:57.024462 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.024388    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:29:57.083555 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.083447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.109249 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.108563    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq" podStartSLOduration=109.108540134 podStartE2EDuration="1m49.108540134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:57.106847866 +0000 UTC m=+110.095697323" watchObservedRunningTime="2026-07-29 10:29:57.108540134 +0000 UTC m=+110.097389590"
Jul 29 10:29:57.109249 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.108799    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-hjpt2" podStartSLOduration=1.751002697 podStartE2EDuration="21.108788021s" totalImagesPullingTime="19.357785324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:29:57.053992006 +0000 UTC m=+110.042841462" watchObservedRunningTime="2026-07-29 10:29:57.108788021 +0000 UTC m=+110.097637479"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.111809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-metrics-client-ca\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.111867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.111901    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.111945    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.111975    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-sys\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112003    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4plqf\" (UniqueName: \"kubernetes.io/projected/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-api-access-4plqf\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-root\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112075    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-wtmp\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lscxj\" (UniqueName: \"kubernetes.io/projected/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-kube-api-access-lscxj\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112224    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.112244 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-textfile\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.113081 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.112276    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-tls\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.113934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.113629    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.116314 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:57.114195    2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 10:29:57.116314 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:57.114259    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-tls podName:f72d1adb-1bb4-4a92-afb3-362b0aa25aac nodeName:}" failed. No retries permitted until 2026-07-29 10:29:57.614238238 +0000 UTC m=+110.603087686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-rnvrg" (UID: "f72d1adb-1bb4-4a92-afb3-362b0aa25aac") : secret "kube-state-metrics-tls" not found
Jul 29 10:29:57.116314 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.115807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.116314 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.116273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.122713 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.122664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.148058 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.147806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4plqf\" (UniqueName: \"kubernetes.io/projected/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-api-access-4plqf\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.192044 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.191854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.213806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-textfile\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.213850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-tls\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.213912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-metrics-client-ca\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214024    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-sys\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-root\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-wtmp\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.214765 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lscxj\" (UniqueName: \"kubernetes.io/projected/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-kube-api-access-lscxj\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.217051 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-wtmp\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.217051 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214948    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-sys\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.217051 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.214988    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-root\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.217051 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.215598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-textfile\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.217051 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.216433    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-metrics-client-ca\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.217724 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.217678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-accelerators-collector-config\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.218898 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.218869    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.221715 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.221662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-node-exporter-tls\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.255065 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.255027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lscxj\" (UniqueName: \"kubernetes.io/projected/7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89-kube-api-access-lscxj\") pod \"node-exporter-hk5zv\" (UID: \"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89\") " pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.290165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.290017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.525876 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.525707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.545533 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.545501    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hk5zv"
Jul 29 10:29:57.564205 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:57.564158    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7db9ff_b7bd_4f36_9800_ef92b8dd4f89.slice/crio-471e3abfd60f439fe681313679c4a45ffdaf5a31968002e9f4b6c97fe53cbe21 WatchSource:0}: Error finding container 471e3abfd60f439fe681313679c4a45ffdaf5a31968002e9f4b6c97fe53cbe21: Status 404 returned error can't find the container with id 471e3abfd60f439fe681313679c4a45ffdaf5a31968002e9f4b6c97fe53cbe21
Jul 29 10:29:57.618373 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.618311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.623226 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.623000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.624593 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.624562    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1adb-1bb4-4a92-afb3-362b0aa25aac-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rnvrg\" (UID: \"f72d1adb-1bb4-4a92-afb3-362b0aa25aac\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.816346 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.816121    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"
Jul 29 10:29:57.845974 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.845904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.996891 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.996854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:57.998114 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:57.998088    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 10:29:58.018659 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.018625    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021677    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021712    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021736    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-web-config\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-594qk\" (UniqueName: \"kubernetes.io/projected/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-kube-api-access-594qk\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-config-out\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.021991    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.022030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.022057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.023841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.022869    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.025139    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.025449    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.025716    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.025950    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-8dlnb\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.026122    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.026348    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.026562    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.026727    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.026903    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 10:29:58.027287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.027085    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 10:29:58.048698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.044836    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg"]
Jul 29 10:29:58.056957 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.056921    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk5zv" event={"ID":"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89","Type":"ContainerStarted","Data":"471e3abfd60f439fe681313679c4a45ffdaf5a31968002e9f4b6c97fe53cbe21"}
Jul 29 10:29:58.122563 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122516    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-config-out\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.122750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.122750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122656    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.122750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.122750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.122750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122893    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-web-config\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.123009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.122991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-594qk\" (UniqueName: \"kubernetes.io/projected/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-kube-api-access-594qk\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.126846 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:29:58.124901    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-trusted-ca-bundle podName:4c5072e6-2a3f-42f3-a41f-ac89a4544ec2 nodeName:}" failed. No retries permitted until 2026-07-29 10:29:58.6248716 +0000 UTC m=+111.613721048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4c5072e6-2a3f-42f3-a41f-ac89a4544ec2") : configmap references non-existent config key: ca-bundle.crt
Jul 29 10:29:58.126846 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.125550    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.126846 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.125933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.144104 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.144053    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-web-config\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.145087 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.145057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-594qk\" (UniqueName: \"kubernetes.io/projected/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-kube-api-access-594qk\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.146603 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.146548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.146603 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.146569    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.147442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.147045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-config-out\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.147442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.147368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-config-volume\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.147442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.147419    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.147773 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.147728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.148486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.148445    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.148579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.148455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.166243 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.166206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:58.629524 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.629428    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.631312 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.631255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c5072e6-2a3f-42f3-a41f-ac89a4544ec2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.649493 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.649144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 10:29:58.851264 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.851222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:58.918971 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.918825    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 10:29:58.933866 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:58.933830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:59.065015 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.064975    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg" event={"ID":"f72d1adb-1bb4-4a92-afb3-362b0aa25aac","Type":"ContainerStarted","Data":"29a6f0587d2fec275f6df020661de2a062a9f1c237826e0515afb72159391699"}
Jul 29 10:29:59.544950 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:29:59.544679    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5072e6_2a3f_42f3_a41f_ac89a4544ec2.slice/crio-1891f94ff40a0fc424f753acad977ed76066766da24632d8666a140693bf178d WatchSource:0}: Error finding container 1891f94ff40a0fc424f753acad977ed76066766da24632d8666a140693bf178d: Status 404 returned error can't find the container with id 1891f94ff40a0fc424f753acad977ed76066766da24632d8666a140693bf178d
Jul 29 10:29:59.787651 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.787588    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:59.839910 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.839415    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64f8f5c957-zpks8"]
Jul 29 10:29:59.856359 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.856310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.857437 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.857378    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64f8f5c957-zpks8"]
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.859830    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-n2shs\""
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.860049    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.860249    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.860269    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.860307    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.860518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 10:29:59.860779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.860539    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eg46bvovsdkar\""
Jul 29 10:29:59.867738 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.867501    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:59.943303 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.943265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.950777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcvbk\" (UniqueName: \"kubernetes.io/projected/bfca97cd-7242-42f8-9554-38467634e9a4-kube-api-access-xcvbk\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.950830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.950870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.950900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.950944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-tls\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.950988    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.951051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfca97cd-7242-42f8-9554-38467634e9a4-metrics-client-ca\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:29:59.951165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:29:59.951077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-grpc-tls\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052126 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052362 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052169    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfca97cd-7242-42f8-9554-38467634e9a4-metrics-client-ca\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052362 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052200    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-grpc-tls\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052362 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcvbk\" (UniqueName: \"kubernetes.io/projected/bfca97cd-7242-42f8-9554-38467634e9a4-kube-api-access-xcvbk\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052362 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052261    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052362 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052304    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052362 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.052687 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.052397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-tls\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.054582 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.054523    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfca97cd-7242-42f8-9554-38467634e9a4-metrics-client-ca\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.059075 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.058755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.063407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.063340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-tls\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.065547 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.065493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.066200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.066116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.066875 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.066831    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-grpc-tls\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.068109 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.067366    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfca97cd-7242-42f8-9554-38467634e9a4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.071065 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.071014    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcvbk\" (UniqueName: \"kubernetes.io/projected/bfca97cd-7242-42f8-9554-38467634e9a4-kube-api-access-xcvbk\") pod \"thanos-querier-64f8f5c957-zpks8\" (UID: \"bfca97cd-7242-42f8-9554-38467634e9a4\") " pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.076227 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.076166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"1891f94ff40a0fc424f753acad977ed76066766da24632d8666a140693bf178d"}
Jul 29 10:30:00.174148 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.173778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:00.218521 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.218418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:00.726386 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.726347    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c48fd7547-8z9mq"]
Jul 29 10:30:00.845070 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.845028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:00.939297 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.939259    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd78686d-ths5q"]
Jul 29 10:30:00.944719 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:00.944419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:01.302775 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.302738    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"]
Jul 29 10:30:01.386163 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.386121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"]
Jul 29 10:30:01.386343 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.386276    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.390148 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.390122    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 10:30:01.391557 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.391373    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-c4kmz\""
Jul 29 10:30:01.391557 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.391424    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6gnqaa28i1b09\""
Jul 29 10:30:01.391752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.391652    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 10:30:01.391872 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.391856    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 10:30:01.392223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.392198    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 10:30:01.467450 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-secret-metrics-server-client-certs\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.467683 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr6cn\" (UniqueName: \"kubernetes.io/projected/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-kube-api-access-xr6cn\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.467683 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467587    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.467683 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-metrics-server-audit-profiles\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.467683 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-audit-log\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.467880 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467714    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-secret-metrics-server-tls\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.467880 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.467749    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-client-ca-bundle\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.568967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-secret-metrics-server-client-certs\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.569029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr6cn\" (UniqueName: \"kubernetes.io/projected/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-kube-api-access-xr6cn\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.569074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.569107    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-metrics-server-audit-profiles\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.569134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-audit-log\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.569178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-secret-metrics-server-tls\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.569223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.569211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-client-ca-bundle\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.570296 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.570163    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.571149 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.570962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-metrics-server-audit-profiles\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.571149 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.571112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-audit-log\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.575268 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.575221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-secret-metrics-server-tls\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.576549 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.576294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-client-ca-bundle\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.576549 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.576510    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-secret-metrics-server-client-certs\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.580709 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.579077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr6cn\" (UniqueName: \"kubernetes.io/projected/6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa-kube-api-access-xr6cn\") pod \"metrics-server-fdd69bcfc-f9mxl\" (UID: \"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa\") " pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.584191 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.584115    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"]
Jul 29 10:30:01.643656 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.643617    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"]
Jul 29 10:30:01.643839 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.643759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.646929 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.646895    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-t8n5q\""
Jul 29 10:30:01.647170 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.646896    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 10:30:01.670190 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.670152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5c8e26e8-735b-4b82-9174-fafed08413a0-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4wjsj\" (UID: \"5c8e26e8-735b-4b82-9174-fafed08413a0\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.670457 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.670228    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sppr\" (UniqueName: \"kubernetes.io/projected/5c8e26e8-735b-4b82-9174-fafed08413a0-kube-api-access-2sppr\") pod \"monitoring-plugin-68b7fc9574-4wjsj\" (UID: \"5c8e26e8-735b-4b82-9174-fafed08413a0\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.700058 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.699960    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:01.710550 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.710459    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:01.771676 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.771606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sppr\" (UniqueName: \"kubernetes.io/projected/5c8e26e8-735b-4b82-9174-fafed08413a0-kube-api-access-2sppr\") pod \"monitoring-plugin-68b7fc9574-4wjsj\" (UID: \"5c8e26e8-735b-4b82-9174-fafed08413a0\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.771866 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.771758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5c8e26e8-735b-4b82-9174-fafed08413a0-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4wjsj\" (UID: \"5c8e26e8-735b-4b82-9174-fafed08413a0\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.779923 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.779880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5c8e26e8-735b-4b82-9174-fafed08413a0-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4wjsj\" (UID: \"5c8e26e8-735b-4b82-9174-fafed08413a0\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.787223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.787176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sppr\" (UniqueName: \"kubernetes.io/projected/5c8e26e8-735b-4b82-9174-fafed08413a0-kube-api-access-2sppr\") pod \"monitoring-plugin-68b7fc9574-4wjsj\" (UID: \"5c8e26e8-735b-4b82-9174-fafed08413a0\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:01.807702 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.807652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:01.886829 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.886698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:01.958107 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:01.957977    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:02.134460 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.134423    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-88b5d58c7-g888w"]
Jul 29 10:30:02.183800 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.183718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-88b5d58c7-g888w"]
Jul 29 10:30:02.183969 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.183850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.187551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.187335    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 10:30:02.187551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.187358    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-flc5t\""
Jul 29 10:30:02.187551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.187374    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 10:30:02.187551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.187367    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 10:30:02.187551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.187522    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 10:30:02.187901 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.187606    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 10:30:02.192223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.192192    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 10:30:02.277456 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-serving-certs-ca-bundle\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277456 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-federate-client-tls\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6p5g\" (UniqueName: \"kubernetes.io/projected/517d370f-445f-464f-bf88-aa374d8852db-kube-api-access-m6p5g\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-telemeter-client-tls\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-telemeter-trusted-ca-bundle\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277699    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-secret-telemeter-client\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277882 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.277882 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.277748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-metrics-client-ca\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.378724 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.378681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-telemeter-client-tls\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.378891 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.378740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-telemeter-trusted-ca-bundle\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.378936 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.378917    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-secret-telemeter-client\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.378972 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.378950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.379025 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.378990    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-metrics-client-ca\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.379107 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.379088    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-serving-certs-ca-bundle\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.379163 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.379122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-federate-client-tls\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.379224 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.379167    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6p5g\" (UniqueName: \"kubernetes.io/projected/517d370f-445f-464f-bf88-aa374d8852db-kube-api-access-m6p5g\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.379808 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.379782    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-metrics-client-ca\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.379987 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.379808    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-serving-certs-ca-bundle\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.380094 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.379845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/517d370f-445f-464f-bf88-aa374d8852db-telemeter-trusted-ca-bundle\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.381882 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.381856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-telemeter-client-tls\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.382050 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.382023    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.382169 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.382119    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-secret-telemeter-client\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.383237 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.383215    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/517d370f-445f-464f-bf88-aa374d8852db-federate-client-tls\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.389960 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.389925    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6p5g\" (UniqueName: \"kubernetes.io/projected/517d370f-445f-464f-bf88-aa374d8852db-kube-api-access-m6p5g\") pod \"telemeter-client-88b5d58c7-g888w\" (UID: \"517d370f-445f-464f-bf88-aa374d8852db\") " pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.497145 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.497089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w"
Jul 29 10:30:02.798530 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.798449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:02.894729 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:02.894690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:03.009702 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.009663    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9c9666fd-qpjl7"]
Jul 29 10:30:03.020614 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.020575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.030090 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.030051    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 10:30:03.030480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.030398    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c9666fd-qpjl7"]
Jul 29 10:30:03.052480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.052367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:03.087992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.087941    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-service-ca\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.088315 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.088279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-oauth-serving-cert\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.088468 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.088361    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-serving-cert\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.088468 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.088408    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-oauth-config\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.088594 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.088502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-trusted-ca-bundle\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.088594 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.088546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-config\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.088594 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.088574    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89v8p\" (UniqueName: \"kubernetes.io/projected/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-kube-api-access-89v8p\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189274 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-service-ca\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-oauth-serving-cert\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189347    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-serving-cert\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-oauth-config\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-trusted-ca-bundle\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-config\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.189566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.189507    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89v8p\" (UniqueName: \"kubernetes.io/projected/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-kube-api-access-89v8p\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.190166 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.190142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-oauth-serving-cert\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.190827 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.190765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-service-ca\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.191493 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.191420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-config\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.191850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.191822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-trusted-ca-bundle\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.193803 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.193780    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-serving-cert\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.194060 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.194041    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-oauth-config\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.198493 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.198452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89v8p\" (UniqueName: \"kubernetes.io/projected/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-kube-api-access-89v8p\") pod \"console-9c9666fd-qpjl7\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") " pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.337718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.337627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:03.992770 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:03.992719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:04.060052 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:04.060008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:05.663709 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:05.663663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:05.764828 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:05.764790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:06.089552 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:06.089511    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64f8f5c957-zpks8"]
Jul 29 10:30:06.103113 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:06.103081    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfca97cd_7242_42f8_9554_38467634e9a4.slice/crio-5fa4e5a426d73e7b95dd60bde2866009d7bec942e7c60aa648f749e0d89cc554 WatchSource:0}: Error finding container 5fa4e5a426d73e7b95dd60bde2866009d7bec942e7c60aa648f749e0d89cc554: Status 404 returned error can't find the container with id 5fa4e5a426d73e7b95dd60bde2866009d7bec942e7c60aa648f749e0d89cc554
Jul 29 10:30:06.280600 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:06.280569    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-88b5d58c7-g888w"]
Jul 29 10:30:06.429266 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:06.429240    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517d370f_445f_464f_bf88_aa374d8852db.slice/crio-edd5c0eba2c7c7da8011b68a1bed44fdec56a337eb68953e483d25aa882c13a7 WatchSource:0}: Error finding container edd5c0eba2c7c7da8011b68a1bed44fdec56a337eb68953e483d25aa882c13a7: Status 404 returned error can't find the container with id edd5c0eba2c7c7da8011b68a1bed44fdec56a337eb68953e483d25aa882c13a7
Jul 29 10:30:06.509108 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:06.508947    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"]
Jul 29 10:30:06.514413 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:06.514120    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8e26e8_735b_4b82_9174_fafed08413a0.slice/crio-2090c5cf96d207dfc4c89b88e96744b74897a29b606f8db554d17f3ba2e15cf6 WatchSource:0}: Error finding container 2090c5cf96d207dfc4c89b88e96744b74897a29b606f8db554d17f3ba2e15cf6: Status 404 returned error can't find the container with id 2090c5cf96d207dfc4c89b88e96744b74897a29b606f8db554d17f3ba2e15cf6
Jul 29 10:30:06.516889 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:06.516866    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9c9666fd-qpjl7"]
Jul 29 10:30:06.518477 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:06.518413    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"]
Jul 29 10:30:06.522872 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:06.522554    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6669f13e_d4be_4e7a_b246_5c1f0b2dc8aa.slice/crio-af247800621a333844ee9dddf1fe4c723eedae609a958df2a6dea6a09f4b6553 WatchSource:0}: Error finding container af247800621a333844ee9dddf1fe4c723eedae609a958df2a6dea6a09f4b6553: Status 404 returned error can't find the container with id af247800621a333844ee9dddf1fe4c723eedae609a958df2a6dea6a09f4b6553
Jul 29 10:30:06.522998 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:06.522873    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbd0b8f_6a3e_4ace_b0ec_1981045f140a.slice/crio-fa0b484b05983fca3d05047e3746c582e7b92caeb9abfbe0def5d1bae9e8a139 WatchSource:0}: Error finding container fa0b484b05983fca3d05047e3746c582e7b92caeb9abfbe0def5d1bae9e8a139: Status 404 returned error can't find the container with id fa0b484b05983fca3d05047e3746c582e7b92caeb9abfbe0def5d1bae9e8a139
Jul 29 10:30:07.121342 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.120941    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd78686d-ths5q" event={"ID":"1d83814a-604e-43ba-ae35-538cf7dce61c","Type":"ContainerStarted","Data":"87ff243e5a206dc6f5e25334e26ba56d2198e3e5d5ab89023c9a0bf30dac7f21"}
Jul 29 10:30:07.125711 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.125575    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg" event={"ID":"f72d1adb-1bb4-4a92-afb3-362b0aa25aac","Type":"ContainerStarted","Data":"5c1d7b874f04cb20c8270660b2ebb51035351a7d03088b56abd6fe49eabfac4a"}
Jul 29 10:30:07.125711 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.125643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg" event={"ID":"f72d1adb-1bb4-4a92-afb3-362b0aa25aac","Type":"ContainerStarted","Data":"8f166c867e54e2d12ae7d1ab34aa8f67af5ef2380b3fb468a8a0c408c83a2529"}
Jul 29 10:30:07.125711 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.125671    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg" event={"ID":"f72d1adb-1bb4-4a92-afb3-362b0aa25aac","Type":"ContainerStarted","Data":"68334de0cf5a22726c3a76beadebadc1e3fdcad68de81674068748de216f883f"}
Jul 29 10:30:07.127799 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.127701    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"5fa4e5a426d73e7b95dd60bde2866009d7bec942e7c60aa648f749e0d89cc554"}
Jul 29 10:30:07.129812 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.129782    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c9666fd-qpjl7" event={"ID":"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a","Type":"ContainerStarted","Data":"bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5"}
Jul 29 10:30:07.129968 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.129835    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c9666fd-qpjl7" event={"ID":"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a","Type":"ContainerStarted","Data":"fa0b484b05983fca3d05047e3746c582e7b92caeb9abfbe0def5d1bae9e8a139"}
Jul 29 10:30:07.132959 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.132898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86nmd" event={"ID":"c44085eb-1b3d-4591-9baf-d9f38d5f47a2","Type":"ContainerStarted","Data":"efe551f33eab25717f8977dcdeab5aac84caf35b405b1e29f10a4fd5a8179ea0"}
Jul 29 10:30:07.134314 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.134284    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl" event={"ID":"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa","Type":"ContainerStarted","Data":"af247800621a333844ee9dddf1fe4c723eedae609a958df2a6dea6a09f4b6553"}
Jul 29 10:30:07.137725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.137018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zbcld" event={"ID":"22601efc-2958-474f-8590-3725796c9e2c","Type":"ContainerStarted","Data":"19a11cb62e1a4a4db848595a817e8972c04832d8ccc3403951e7fb359bbaca25"}
Jul 29 10:30:07.137725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.137076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zbcld" event={"ID":"22601efc-2958-474f-8590-3725796c9e2c","Type":"ContainerStarted","Data":"e82578b7e78f98827e9800fd8d350cc9cf29de757ea6ad3fa2109b0af87247da"}
Jul 29 10:30:07.137725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.137626    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zbcld"
Jul 29 10:30:07.140571 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.139862    2562 generic.go:356] "Generic (PLEG): container finished" podID="7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89" containerID="023ff90f74e9e1fcfd0143e7a2373f3aa8a4fc338f34505fb2d7470fef82427e" exitCode=0
Jul 29 10:30:07.140571 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.140475    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk5zv" event={"ID":"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89","Type":"ContainerDied","Data":"023ff90f74e9e1fcfd0143e7a2373f3aa8a4fc338f34505fb2d7470fef82427e"}
Jul 29 10:30:07.142509 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.142219    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd78686d-ths5q" podStartSLOduration=9.78599961 podStartE2EDuration="19.142203814s" totalImagesPullingTime="9.356204204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:07.140774969 +0000 UTC m=+120.129624426" watchObservedRunningTime="2026-07-29 10:30:07.142203814 +0000 UTC m=+120.131053270"
Jul 29 10:30:07.143413 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.143361    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj" event={"ID":"5c8e26e8-735b-4b82-9174-fafed08413a0","Type":"ContainerStarted","Data":"2090c5cf96d207dfc4c89b88e96744b74897a29b606f8db554d17f3ba2e15cf6"}
Jul 29 10:30:07.145918 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.145890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w" event={"ID":"517d370f-445f-464f-bf88-aa374d8852db","Type":"ContainerStarted","Data":"edd5c0eba2c7c7da8011b68a1bed44fdec56a337eb68953e483d25aa882c13a7"}
Jul 29 10:30:07.148197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.148161    2562 generic.go:356] "Generic (PLEG): container finished" podID="4c5072e6-2a3f-42f3-a41f-ac89a4544ec2" containerID="39475f3f57c67d7d2778bc497ed098d96c8bb9fb36456e8785976785fe89e270" exitCode=0
Jul 29 10:30:07.148370 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.148235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerDied","Data":"39475f3f57c67d7d2778bc497ed098d96c8bb9fb36456e8785976785fe89e270"}
Jul 29 10:30:07.186252 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.185688    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rnvrg" podStartSLOduration=3.313370386 podStartE2EDuration="11.185665824s" totalImagesPullingTime="7.872295438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:07.18384188 +0000 UTC m=+120.172691333" watchObservedRunningTime="2026-07-29 10:30:07.185665824 +0000 UTC m=+120.174515282"
Jul 29 10:30:07.186252 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.185926    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zbcld" podStartSLOduration=74.847876815 podStartE2EDuration="1m24.185915544s" totalImagesPullingTime="9.338038729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:07.163185002 +0000 UTC m=+120.152034458" watchObservedRunningTime="2026-07-29 10:30:07.185915544 +0000 UTC m=+120.174765001"
Jul 29 10:30:07.208482 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:07.207111    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9c9666fd-qpjl7" podStartSLOduration=5.207091168 podStartE2EDuration="5.207091168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:07.205665823 +0000 UTC m=+120.194515281" watchObservedRunningTime="2026-07-29 10:30:07.207091168 +0000 UTC m=+120.195940625"
Jul 29 10:30:08.156106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:08.156010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk5zv" event={"ID":"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89","Type":"ContainerStarted","Data":"5d1ab46d670b1ab3634833e94c421a2cb7476c3176211af35c3674f5ed63645c"}
Jul 29 10:30:08.156106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:08.156079    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hk5zv" event={"ID":"7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89","Type":"ContainerStarted","Data":"2ba47b2532abebac8c9c17b411748adeaaa12f991d5a967a1fd9b7206ce44d6c"}
Jul 29 10:30:08.184444 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:08.184357    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hk5zv" podStartSLOduration=4.806342089 podStartE2EDuration="12.18433931s" totalImagesPullingTime="7.292413279s" totalInitContainerRuntime="85.583942ms" isStatefulPod=true podCreationTimestamp="2026-07-29 10:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:08.182608768 +0000 UTC m=+121.171458227" watchObservedRunningTime="2026-07-29 10:30:08.18433931 +0000 UTC m=+121.173188768"
Jul 29 10:30:09.150517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:09.150471    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:30:10.665901 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:10.665863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:10.769498 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:10.769462    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:10.938242 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:10.938157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:11.137127 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:11.137095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:11.240725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:11.240689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:13.338492 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:13.338454    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:13.338899 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:13.338521    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:13.343246 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:13.343224    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:14.183442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.183300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"29570dce526b9dd3ca041d0bb5320c91cb0da22fe80b6637d457142abcb8cec2"}
Jul 29 10:30:14.186036 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.186001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-86nmd" event={"ID":"c44085eb-1b3d-4591-9baf-d9f38d5f47a2","Type":"ContainerStarted","Data":"84a56f3cc42cfbba389c4aab3980662b7ab45a1efdd02e77cec49f0962cd26c0"}
Jul 29 10:30:14.188598 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.188545    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl" event={"ID":"6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa","Type":"ContainerStarted","Data":"0b36d8805b90280d0f226003fef097d265a538ad0092e7304a5e54cb89e7c784"}
Jul 29 10:30:14.190709 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.190629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj" event={"ID":"5c8e26e8-735b-4b82-9174-fafed08413a0","Type":"ContainerStarted","Data":"eab65a97c703d84b2f3ddef8be3e5c716916389132d793cfdd2442fceb062d77"}
Jul 29 10:30:14.191084 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.191040    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:14.193056 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.192958    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w" event={"ID":"517d370f-445f-464f-bf88-aa374d8852db","Type":"ContainerStarted","Data":"e73cec97518dd02c49908045ed1ca133bac78a5f99e70d64b0676b24a4f54803"}
Jul 29 10:30:14.193056 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.193015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w" event={"ID":"517d370f-445f-464f-bf88-aa374d8852db","Type":"ContainerStarted","Data":"7a5c54d81394395604514ff93d2b15f5a868a39469df5f363667278c7887ee56"}
Jul 29 10:30:14.199359 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.199313    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj"
Jul 29 10:30:14.201836 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.201799    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"697955ab98dcf168ab34c470be891c9844cb170215d215bb0575726f1436801a"}
Jul 29 10:30:14.208954 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.208072    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:14.239351 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.239268    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-86nmd" podStartSLOduration=33.674434453 podStartE2EDuration="1m3.239252393s" totalImagesPullingTime="29.56481794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 10:29:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:14.208309281 +0000 UTC m=+127.197158737" watchObservedRunningTime="2026-07-29 10:30:14.239252393 +0000 UTC m=+127.228101849"
Jul 29 10:30:14.310141 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.309994    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4wjsj" podStartSLOduration=6.000744244 podStartE2EDuration="13.309953883s" totalImagesPullingTime="7.309209639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:14.277144567 +0000 UTC m=+127.265994030" watchObservedRunningTime="2026-07-29 10:30:14.309953883 +0000 UTC m=+127.298803342"
Jul 29 10:30:14.311401 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.311365    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl" podStartSLOduration=6.01153688 podStartE2EDuration="13.311348095s" totalImagesPullingTime="7.299811215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:14.30972309 +0000 UTC m=+127.298572547" watchObservedRunningTime="2026-07-29 10:30:14.311348095 +0000 UTC m=+127.300197559"
Jul 29 10:30:14.456370 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.456316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:14.567737 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:14.567706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:15.209871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.209829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w" event={"ID":"517d370f-445f-464f-bf88-aa374d8852db","Type":"ContainerStarted","Data":"16dca5d8ba2ac32010efbb8268b3fdc698b7eb61c1ce92221b9f5787ef9e7990"}
Jul 29 10:30:15.213377 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.213344    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"03fee8bd1471fd85a2ca83291edbec57e6381bfd0b26c2c1d91e6dfb33f244f1"}
Jul 29 10:30:15.213542 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.213424    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"59d7ec1a0f9bc59d895fa13f366190b79c9294529d15096607c76e2f39891073"}
Jul 29 10:30:15.213542 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.213472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"31b3f10ac4bbe3b1e125b9d512c3ce969f4808bf32e6cb34faa2b0c4a6f9f979"}
Jul 29 10:30:15.213542 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.213510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"79145bc43e19e05bf7472cd73e311c6bf0cc63a93a73c813899188f25be8bc78"}
Jul 29 10:30:15.215893 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.215777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"b1f4ad11f0c33be31b08edf7c7500ba68ba50af5b7612b9a5ee77d49bc3f5373"}
Jul 29 10:30:15.215893 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.215856    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"54e8b62b05b3117053d6d01e66d8119a400a29f88f487c3a6115c03a5ecfcbd5"}
Jul 29 10:30:15.251675 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:15.251602    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-88b5d58c7-g888w" podStartSLOduration=5.894570336 podStartE2EDuration="13.251579487s" totalImagesPullingTime="7.357009151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:15.250884432 +0000 UTC m=+128.239733889" watchObservedRunningTime="2026-07-29 10:30:15.251579487 +0000 UTC m=+128.240428944"
Jul 29 10:30:16.222725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.222686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"ba243dea41c055b215358292c15317a3a81f4fd09c3734a47d8d24debea9dcb9"}
Jul 29 10:30:16.223209 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.222756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"168a6f16a2b5a1127e120eb259b193b1055a303cfc34d52bc7f0cbadf494284a"}
Jul 29 10:30:16.223209 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.222789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" event={"ID":"bfca97cd-7242-42f8-9554-38467634e9a4","Type":"ContainerStarted","Data":"8042f0535bf59340d0944802fa858780558bf79474a20e9c15926fbbf05ca853"}
Jul 29 10:30:16.223209 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.222820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:16.225905 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.225872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4c5072e6-2a3f-42f3-a41f-ac89a4544ec2","Type":"ContainerStarted","Data":"3d1916f5c8335722843c54e95af493c6d196095fac1d37584a70740f62c09515"}
Jul 29 10:30:16.249681 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.249578    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8" podStartSLOduration=8.056568857 podStartE2EDuration="17.2495624s" totalImagesPullingTime="9.192993543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:16.248698328 +0000 UTC m=+129.237547786" watchObservedRunningTime="2026-07-29 10:30:16.2495624 +0000 UTC m=+129.238411855"
Jul 29 10:30:16.283708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.283652    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.425510731 podStartE2EDuration="19.283634076s" totalImagesPullingTime="14.836353216s" totalInitContainerRuntime="21.770129ms" isStatefulPod=false podCreationTimestamp="2026-07-29 10:29:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:16.281298214 +0000 UTC m=+129.270147707" watchObservedRunningTime="2026-07-29 10:30:16.283634076 +0000 UTC m=+129.272483531"
Jul 29 10:30:16.369736 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.369699    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c94bd674-zvnw4"]
Jul 29 10:30:16.386984 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.386942    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c94bd674-zvnw4"]
Jul 29 10:30:16.387155 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.387081    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.421221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.421190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:16.497924 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.497883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:16.534480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534384    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-service-ca\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.534480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534439    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv9vz\" (UniqueName: \"kubernetes.io/projected/caaf7193-9d0f-4103-8c23-8be8421660ca-kube-api-access-cv9vz\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.534669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-oauth-serving-cert\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.534669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534521    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-trusted-ca-bundle\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.534669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-serving-cert\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.534669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534655    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-console-config\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.534934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.534688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-oauth-config\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635202 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-trusted-ca-bundle\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635408 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-serving-cert\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635408 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-console-config\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635507 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-oauth-config\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635507 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635496    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-service-ca\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635635 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv9vz\" (UniqueName: \"kubernetes.io/projected/caaf7193-9d0f-4103-8c23-8be8421660ca-kube-api-access-cv9vz\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.635721 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.635703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-oauth-serving-cert\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.636077 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.636053    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-console-config\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.636257 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.636221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-service-ca\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.636404 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.636383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-oauth-serving-cert\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.636453 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.636408    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-trusted-ca-bundle\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.638122 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.638104    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-serving-cert\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.638200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.638133    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-oauth-config\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.644402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.644382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv9vz\" (UniqueName: \"kubernetes.io/projected/caaf7193-9d0f-4103-8c23-8be8421660ca-kube-api-access-cv9vz\") pod \"console-78c94bd674-zvnw4\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") " pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.697513 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.697472    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:16.861001 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:16.860974    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c94bd674-zvnw4"]
Jul 29 10:30:16.863640 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:16.863601    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaaf7193_9d0f_4103_8c23_8be8421660ca.slice/crio-b70a02878068d7309989557c1b15278505a3369040bb626a268a2e58e63a16c4 WatchSource:0}: Error finding container b70a02878068d7309989557c1b15278505a3369040bb626a268a2e58e63a16c4: Status 404 returned error can't find the container with id b70a02878068d7309989557c1b15278505a3369040bb626a268a2e58e63a16c4
Jul 29 10:30:17.158519 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.158436    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zbcld"
Jul 29 10:30:17.234811 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.234773    2562 generic.go:356] "Generic (PLEG): container finished" podID="7ab95840-c1d9-48ef-a841-8a7bf578ec09" containerID="d8d903f63db2d9d678f6ef7050a46b10da9be5c752881ce5279d61948a0dea82" exitCode=0
Jul 29 10:30:17.235309 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.234848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld" event={"ID":"7ab95840-c1d9-48ef-a841-8a7bf578ec09","Type":"ContainerDied","Data":"d8d903f63db2d9d678f6ef7050a46b10da9be5c752881ce5279d61948a0dea82"}
Jul 29 10:30:17.235402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.235308    2562 scope.go:117] "RemoveContainer" containerID="d8d903f63db2d9d678f6ef7050a46b10da9be5c752881ce5279d61948a0dea82"
Jul 29 10:30:17.236544 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.236524    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c94bd674-zvnw4" event={"ID":"caaf7193-9d0f-4103-8c23-8be8421660ca","Type":"ContainerStarted","Data":"66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20"}
Jul 29 10:30:17.236637 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.236570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c94bd674-zvnw4" event={"ID":"caaf7193-9d0f-4103-8c23-8be8421660ca","Type":"ContainerStarted","Data":"b70a02878068d7309989557c1b15278505a3369040bb626a268a2e58e63a16c4"}
Jul 29 10:30:17.276884 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.276832    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78c94bd674-zvnw4" podStartSLOduration=1.276814596 podStartE2EDuration="1.276814596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:17.275913689 +0000 UTC m=+130.264763168" watchObservedRunningTime="2026-07-29 10:30:17.276814596 +0000 UTC m=+130.265664053"
Jul 29 10:30:17.444608 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.444519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:30:17.446866 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.446836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f1afb4-3c15-4858-8291-48c7603f443a-metrics-certs\") pod \"network-metrics-daemon-hrntc\" (UID: \"79f1afb4-3c15-4858-8291-48c7603f443a\") " pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:30:17.715970 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.715889    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l8nkz\""
Jul 29 10:30:17.723548 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.723239    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrntc"
Jul 29 10:30:17.885435 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:17.885356    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hrntc"]
Jul 29 10:30:17.887979 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:30:17.887951    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f1afb4_3c15_4858_8291_48c7603f443a.slice/crio-4893b7dcc8c973c7a995bd354ca948e698157bd64acd84140b99c010c0a03b7b WatchSource:0}: Error finding container 4893b7dcc8c973c7a995bd354ca948e698157bd64acd84140b99c010c0a03b7b: Status 404 returned error can't find the container with id 4893b7dcc8c973c7a995bd354ca948e698157bd64acd84140b99c010c0a03b7b
Jul 29 10:30:18.242957 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:18.242913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-bkxld" event={"ID":"7ab95840-c1d9-48ef-a841-8a7bf578ec09","Type":"ContainerStarted","Data":"cbf2e510212de45b3dc22515049bc7e1826418d769cab293677edac238207a6b"}
Jul 29 10:30:18.244462 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:18.244417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrntc" event={"ID":"79f1afb4-3c15-4858-8291-48c7603f443a","Type":"ContainerStarted","Data":"4893b7dcc8c973c7a995bd354ca948e698157bd64acd84140b99c010c0a03b7b"}
Jul 29 10:30:20.255140 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.255103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrntc" event={"ID":"79f1afb4-3c15-4858-8291-48c7603f443a","Type":"ContainerStarted","Data":"0ce458fc7d786c5b11a53119c0c998290d297484617cc692b9c4c9d294f4f6ba"}
Jul 29 10:30:20.255585 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.255157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrntc" event={"ID":"79f1afb4-3c15-4858-8291-48c7603f443a","Type":"ContainerStarted","Data":"b0da4b6aff9c680d4f3cc5243cc5a1bd920890344ace3d7bed244ecb2607bf2c"}
Jul 29 10:30:20.274105 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.274048    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hrntc" podStartSLOduration=130.821299985 podStartE2EDuration="2m12.2740323s" totalImagesPullingTime="1.452732315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:30:20.272570977 +0000 UTC m=+133.261420434" watchObservedRunningTime="2026-07-29 10:30:20.2740323 +0000 UTC m=+133.262881755"
Jul 29 10:30:20.391054 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.391013    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:20.475205 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.475164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:20.678340 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.678232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:20.732635 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.732611    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:30:20.785267 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:20.785230    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:21.700317 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:21.700273    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:21.700729 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:21.700376    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:22.243439 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:22.243411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64f8f5c957-zpks8"
Jul 29 10:30:25.750772 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:25.750711    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq" podUID="bdb9f707-359d-4e5d-a6bc-6c3da935da90" containerName="registry" containerID="cri-o://73cbf0034a763d52f816b187b972423ee758c3932e230e1a1b6725d9e666d89e" gracePeriod=30
Jul 29 10:30:26.283782 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.283750    2562 generic.go:356] "Generic (PLEG): container finished" podID="bdb9f707-359d-4e5d-a6bc-6c3da935da90" containerID="73cbf0034a763d52f816b187b972423ee758c3932e230e1a1b6725d9e666d89e" exitCode=0
Jul 29 10:30:26.283950 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.283824    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq" event={"ID":"bdb9f707-359d-4e5d-a6bc-6c3da935da90","Type":"ContainerDied","Data":"73cbf0034a763d52f816b187b972423ee758c3932e230e1a1b6725d9e666d89e"}
Jul 29 10:30:26.285299 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.285278    2562 generic.go:356] "Generic (PLEG): container finished" podID="2caed74b-6436-4709-9330-ede74f2cb0c2" containerID="e5df9831d4239708f5f15221049b66246833f26f3e2489ecbad4c3cf44fbf593" exitCode=0
Jul 29 10:30:26.285417 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.285356    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss" event={"ID":"2caed74b-6436-4709-9330-ede74f2cb0c2","Type":"ContainerDied","Data":"e5df9831d4239708f5f15221049b66246833f26f3e2489ecbad4c3cf44fbf593"}
Jul 29 10:30:26.285727 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.285712    2562 scope.go:117] "RemoveContainer" containerID="e5df9831d4239708f5f15221049b66246833f26f3e2489ecbad4c3cf44fbf593"
Jul 29 10:30:26.291038 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.291017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:26.398640 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.398605    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:26.530606 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.530579    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:30:26.632095 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdb9f707-359d-4e5d-a6bc-6c3da935da90-ca-trust-extracted\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632295 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632138    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-certificates\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632295 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632166    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-installation-pull-secrets\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632295 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632195    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlw7w\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-kube-api-access-dlw7w\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632295 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632230    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-bound-sa-token\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632295 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632288    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-image-registry-private-configuration\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632575 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632333    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.632575 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632382    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-trusted-ca\") pod \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\" (UID: \"bdb9f707-359d-4e5d-a6bc-6c3da935da90\") "
Jul 29 10:30:26.633240 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.632872    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:26.633604 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.633570    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:26.635115 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.635086    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:26.635215 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.635183    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:26.635313 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.635288    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:26.635537 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.635506    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-kube-api-access-dlw7w" (OuterVolumeSpecName: "kube-api-access-dlw7w") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "kube-api-access-dlw7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:26.635705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.635682    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:26.641170 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.641144    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb9f707-359d-4e5d-a6bc-6c3da935da90-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bdb9f707-359d-4e5d-a6bc-6c3da935da90" (UID: "bdb9f707-359d-4e5d-a6bc-6c3da935da90"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:30:26.697608 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.697573    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:26.697804 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.697637    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:26.703826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.703793    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:26.733498 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733453    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-trusted-ca\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733498 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733493    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdb9f707-359d-4e5d-a6bc-6c3da935da90-ca-trust-extracted\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733508    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-certificates\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733523    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-installation-pull-secrets\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733539    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlw7w\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-kube-api-access-dlw7w\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733553    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-bound-sa-token\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733567    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdb9f707-359d-4e5d-a6bc-6c3da935da90-image-registry-private-configuration\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:26.733776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:26.733581    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdb9f707-359d-4e5d-a6bc-6c3da935da90-registry-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:27.291247 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.291208    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss" event={"ID":"2caed74b-6436-4709-9330-ede74f2cb0c2","Type":"ContainerStarted","Data":"f9c485a326b6132b3172eaceac7d5b7f66e2729b0d91395d84070fd9637614f4"}
Jul 29 10:30:27.292523 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.292503    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq"
Jul 29 10:30:27.292630 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.292533    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48fd7547-8z9mq" event={"ID":"bdb9f707-359d-4e5d-a6bc-6c3da935da90","Type":"ContainerDied","Data":"64fa466872beb710eab657ee8cd7e5e75676d3253099781e526f85cd7710fcad"}
Jul 29 10:30:27.292630 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.292571    2562 scope.go:117] "RemoveContainer" containerID="73cbf0034a763d52f816b187b972423ee758c3932e230e1a1b6725d9e666d89e"
Jul 29 10:30:27.297617 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.297596    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:30:27.324953 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.324916    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c48fd7547-8z9mq"]
Jul 29 10:30:27.333825 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.333794    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c48fd7547-8z9mq"]
Jul 29 10:30:27.379457 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.379419    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9c9666fd-qpjl7"]
Jul 29 10:30:27.478375 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.478344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:27.509541 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.509511    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdb9f707-359d-4e5d-a6bc-6c3da935da90" path="/var/lib/kubelet/pods/bdb9f707-359d-4e5d-a6bc-6c3da935da90/volumes"
Jul 29 10:30:27.581233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.581145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:27.651162 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:27.651118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:32.148553 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.148511    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cd78686d-ths5q" podUID="1d83814a-604e-43ba-ae35-538cf7dce61c" containerName="console" containerID="cri-o://87ff243e5a206dc6f5e25334e26ba56d2198e3e5d5ab89023c9a0bf30dac7f21" gracePeriod=15
Jul 29 10:30:32.318112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.318078    2562 generic.go:356] "Generic (PLEG): container finished" podID="1d83814a-604e-43ba-ae35-538cf7dce61c" containerID="87ff243e5a206dc6f5e25334e26ba56d2198e3e5d5ab89023c9a0bf30dac7f21" exitCode=0
Jul 29 10:30:32.318253 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.318111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd78686d-ths5q" event={"ID":"1d83814a-604e-43ba-ae35-538cf7dce61c","Type":"ContainerDied","Data":"87ff243e5a206dc6f5e25334e26ba56d2198e3e5d5ab89023c9a0bf30dac7f21"}
Jul 29 10:30:32.399440 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.399367    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:30:32.584199 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584154    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-oauth-serving-cert\") pod \"1d83814a-604e-43ba-ae35-538cf7dce61c\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") "
Jul 29 10:30:32.584199 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584200    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-console-config\") pod \"1d83814a-604e-43ba-ae35-538cf7dce61c\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") "
Jul 29 10:30:32.584486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584257    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-oauth-config\") pod \"1d83814a-604e-43ba-ae35-538cf7dce61c\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") "
Jul 29 10:30:32.584486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584274    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhl2h\" (UniqueName: \"kubernetes.io/projected/1d83814a-604e-43ba-ae35-538cf7dce61c-kube-api-access-jhl2h\") pod \"1d83814a-604e-43ba-ae35-538cf7dce61c\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") "
Jul 29 10:30:32.584486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584306    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-service-ca\") pod \"1d83814a-604e-43ba-ae35-538cf7dce61c\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") "
Jul 29 10:30:32.584486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584353    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-serving-cert\") pod \"1d83814a-604e-43ba-ae35-538cf7dce61c\" (UID: \"1d83814a-604e-43ba-ae35-538cf7dce61c\") "
Jul 29 10:30:32.584730 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584704    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1d83814a-604e-43ba-ae35-538cf7dce61c" (UID: "1d83814a-604e-43ba-ae35-538cf7dce61c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:32.584795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584700    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-console-config" (OuterVolumeSpecName: "console-config") pod "1d83814a-604e-43ba-ae35-538cf7dce61c" (UID: "1d83814a-604e-43ba-ae35-538cf7dce61c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:32.584795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.584775    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-service-ca" (OuterVolumeSpecName: "service-ca") pod "1d83814a-604e-43ba-ae35-538cf7dce61c" (UID: "1d83814a-604e-43ba-ae35-538cf7dce61c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:32.586730 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.586697    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d83814a-604e-43ba-ae35-538cf7dce61c-kube-api-access-jhl2h" (OuterVolumeSpecName: "kube-api-access-jhl2h") pod "1d83814a-604e-43ba-ae35-538cf7dce61c" (UID: "1d83814a-604e-43ba-ae35-538cf7dce61c"). InnerVolumeSpecName "kube-api-access-jhl2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:32.586730 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.586714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1d83814a-604e-43ba-ae35-538cf7dce61c" (UID: "1d83814a-604e-43ba-ae35-538cf7dce61c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:32.586894 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.586767    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1d83814a-604e-43ba-ae35-538cf7dce61c" (UID: "1d83814a-604e-43ba-ae35-538cf7dce61c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:32.685685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.685587    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-service-ca\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:32.685685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.685621    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:32.685685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.685632    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-oauth-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:32.685685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.685641    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1d83814a-604e-43ba-ae35-538cf7dce61c-console-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:32.685685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.685650    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1d83814a-604e-43ba-ae35-538cf7dce61c-console-oauth-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:32.685685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:32.685659    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhl2h\" (UniqueName: \"kubernetes.io/projected/1d83814a-604e-43ba-ae35-538cf7dce61c-kube-api-access-jhl2h\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:33.324991 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:33.324964    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd78686d-ths5q"
Jul 29 10:30:33.324991 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:33.324968    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd78686d-ths5q" event={"ID":"1d83814a-604e-43ba-ae35-538cf7dce61c","Type":"ContainerDied","Data":"b49ca240fc7a3fb3a08f4b807babdc6e94c971f2e62ad6db1613749c000fa023"}
Jul 29 10:30:33.325451 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:33.325052    2562 scope.go:117] "RemoveContainer" containerID="87ff243e5a206dc6f5e25334e26ba56d2198e3e5d5ab89023c9a0bf30dac7f21"
Jul 29 10:30:33.347141 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:33.347108    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd78686d-ths5q"]
Jul 29 10:30:33.350216 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:33.350187    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd78686d-ths5q"]
Jul 29 10:30:33.506564 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:33.506528    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d83814a-604e-43ba-ae35-538cf7dce61c" path="/var/lib/kubelet/pods/1d83814a-604e-43ba-ae35-538cf7dce61c/volumes"
Jul 29 10:30:35.666163 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:35.666125    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:35.733704 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:35.733664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:41.706137 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:41.706105    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:41.709804 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:41.709780    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-fdd69bcfc-f9mxl"
Jul 29 10:30:41.880097 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:41.880057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:48.225125 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:48.225087    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:48.298431 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:48.298395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:30:52.399617 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:52.399575    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9c9666fd-qpjl7" podUID="8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" containerName="console" containerID="cri-o://bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5" gracePeriod=15
Jul 29 10:30:53.182645 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.182607    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:53.244259 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244221    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-oauth-serving-cert\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244466 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244301    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-trusted-ca-bundle\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244466 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244363    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-oauth-config\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244466 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244389    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-serving-cert\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244466 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244412    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-service-ca\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244466 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244445    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89v8p\" (UniqueName: \"kubernetes.io/projected/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-kube-api-access-89v8p\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244720 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244471    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-config\") pod \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\" (UID: \"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a\") "
Jul 29 10:30:53.244789 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244757    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:53.244846 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244762    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:53.244946 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244916    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-service-ca" (OuterVolumeSpecName: "service-ca") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:53.244998 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.244964    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-config" (OuterVolumeSpecName: "console-config") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:30:53.246642 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.246613    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:53.246732 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.246648    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:30:53.246732 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.246662    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-kube-api-access-89v8p" (OuterVolumeSpecName: "kube-api-access-89v8p") pod "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" (UID: "8cbd0b8f-6a3e-4ace-b0ec-1981045f140a"). InnerVolumeSpecName "kube-api-access-89v8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345238    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-trusted-ca-bundle\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345271    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-oauth-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345281    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345293    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-service-ca\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345302    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89v8p\" (UniqueName: \"kubernetes.io/projected/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-kube-api-access-89v8p\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345312    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-console-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.345352 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.345343    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a-oauth-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:30:53.428629 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.428595    2562 generic.go:356] "Generic (PLEG): container finished" podID="8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" containerID="bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5" exitCode=0
Jul 29 10:30:53.429054 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.428643    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9c9666fd-qpjl7"
Jul 29 10:30:53.429054 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.428666    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c9666fd-qpjl7" event={"ID":"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a","Type":"ContainerDied","Data":"bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5"}
Jul 29 10:30:53.429054 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.428707    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9c9666fd-qpjl7" event={"ID":"8cbd0b8f-6a3e-4ace-b0ec-1981045f140a","Type":"ContainerDied","Data":"fa0b484b05983fca3d05047e3746c582e7b92caeb9abfbe0def5d1bae9e8a139"}
Jul 29 10:30:53.429054 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.428726    2562 scope.go:117] "RemoveContainer" containerID="bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5"
Jul 29 10:30:53.437144 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.437118    2562 scope.go:117] "RemoveContainer" containerID="bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5"
Jul 29 10:30:53.437456 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:30:53.437434    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5\": container with ID starting with bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5 not found: ID does not exist" containerID="bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5"
Jul 29 10:30:53.437520 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.437467    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5"} err="failed to get container status \"bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5\": rpc error: code = NotFound desc = could not find container \"bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5\": container with ID starting with bd5c194f61f64222952f0b48232e2840e40799018b701d69416a520ac64b46c5 not found: ID does not exist"
Jul 29 10:30:53.450759 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.450713    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9c9666fd-qpjl7"]
Jul 29 10:30:53.452456 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.452429    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9c9666fd-qpjl7"]
Jul 29 10:30:53.506286 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:30:53.506252    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" path="/var/lib/kubelet/pods/8cbd0b8f-6a3e-4ace-b0ec-1981045f140a/volumes"
Jul 29 10:31:12.939075 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:12.939024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:13.017361 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:13.017300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:24.964522 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:24.964489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:25.024133 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:25.024071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:30.497207 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497166    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5869686cdc-lgz7r"]
Jul 29 10:31:30.497778 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497750    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d83814a-604e-43ba-ae35-538cf7dce61c" containerName="console"
Jul 29 10:31:30.497778 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497778    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d83814a-604e-43ba-ae35-538cf7dce61c" containerName="console"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497797    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" containerName="console"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497805    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" containerName="console"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497846    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb9f707-359d-4e5d-a6bc-6c3da935da90" containerName="registry"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497856    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb9f707-359d-4e5d-a6bc-6c3da935da90" containerName="registry"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497948    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdb9f707-359d-4e5d-a6bc-6c3da935da90" containerName="registry"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497962    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d83814a-604e-43ba-ae35-538cf7dce61c" containerName="console"
Jul 29 10:31:30.497981 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.497972    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cbd0b8f-6a3e-4ace-b0ec-1981045f140a" containerName="console"
Jul 29 10:31:30.501294 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.501261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.509764 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.509730    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5869686cdc-lgz7r"]
Jul 29 10:31:30.542041 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.542008    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:30.612696 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.612659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:30.676188 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-oauth-config\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.676387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676223    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-service-ca\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.676387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676261    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-config\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.676387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-trusted-ca-bundle\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.676387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-serving-cert\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.676638 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-oauth-serving-cert\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.676638 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.676486    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gwtc\" (UniqueName: \"kubernetes.io/projected/36cf0f71-989a-48f7-854c-c69c22e0dd0d-kube-api-access-7gwtc\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.777883 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-oauth-config\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.777883 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-service-ca\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.777883 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-config\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-trusted-ca-bundle\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-serving-cert\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-oauth-serving-cert\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778164 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.777964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gwtc\" (UniqueName: \"kubernetes.io/projected/36cf0f71-989a-48f7-854c-c69c22e0dd0d-kube-api-access-7gwtc\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778683 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.778658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-service-ca\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778775 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.778698    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-config\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.778825 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.778802    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-oauth-serving-cert\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.779170 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.779150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-trusted-ca-bundle\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.780576 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.780547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-serving-cert\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.780674 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.780573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-oauth-config\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.787141 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.787113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gwtc\" (UniqueName: \"kubernetes.io/projected/36cf0f71-989a-48f7-854c-c69c22e0dd0d-kube-api-access-7gwtc\") pod \"console-5869686cdc-lgz7r\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") " pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.812972 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.812935    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:30.957672 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:30.957631    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5869686cdc-lgz7r"]
Jul 29 10:31:30.960819 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:31:30.960782    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36cf0f71_989a_48f7_854c_c69c22e0dd0d.slice/crio-62129157bcb675428f0e9e1b894b41c88ed04d17be4fa0dc468b6aeaee5b1d74 WatchSource:0}: Error finding container 62129157bcb675428f0e9e1b894b41c88ed04d17be4fa0dc468b6aeaee5b1d74: Status 404 returned error can't find the container with id 62129157bcb675428f0e9e1b894b41c88ed04d17be4fa0dc468b6aeaee5b1d74
Jul 29 10:31:31.603014 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:31.602972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5869686cdc-lgz7r" event={"ID":"36cf0f71-989a-48f7-854c-c69c22e0dd0d","Type":"ContainerStarted","Data":"b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53"}
Jul 29 10:31:31.603457 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:31.603034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5869686cdc-lgz7r" event={"ID":"36cf0f71-989a-48f7-854c-c69c22e0dd0d","Type":"ContainerStarted","Data":"62129157bcb675428f0e9e1b894b41c88ed04d17be4fa0dc468b6aeaee5b1d74"}
Jul 29 10:31:31.623674 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:31.623596    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5869686cdc-lgz7r" podStartSLOduration=1.623577418 podStartE2EDuration="1.623577418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:31:31.621149058 +0000 UTC m=+204.609998519" watchObservedRunningTime="2026-07-29 10:31:31.623577418 +0000 UTC m=+204.612426875"
Jul 29 10:31:35.665075 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:35.665033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:35.765665 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:35.765629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:40.813478 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:40.813419    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:40.813478 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:40.813500    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:40.818591 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:40.818560    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:41.645279 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:41.645250    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:31:41.697083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:41.697034    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c94bd674-zvnw4"]
Jul 29 10:31:41.799618 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:41.799580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:41.904582 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:41.904489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:31:42.001652 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:31:42.001599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:06.719457 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:06.719394    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78c94bd674-zvnw4" podUID="caaf7193-9d0f-4103-8c23-8be8421660ca" containerName="console" containerID="cri-o://66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20" gracePeriod=15
Jul 29 10:32:06.966913 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:06.966887    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:32:07.099637 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-trusted-ca-bundle\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.099907 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099666    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv9vz\" (UniqueName: \"kubernetes.io/projected/caaf7193-9d0f-4103-8c23-8be8421660ca-kube-api-access-cv9vz\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.099907 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099685    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-oauth-config\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.099907 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-oauth-serving-cert\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.099907 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099739    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-serving-cert\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.099907 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099800    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-service-ca\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.099907 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.099830    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-console-config\") pod \"caaf7193-9d0f-4103-8c23-8be8421660ca\" (UID: \"caaf7193-9d0f-4103-8c23-8be8421660ca\") "
Jul 29 10:32:07.100231 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.100115    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:32:07.100363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.100307    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:32:07.100432 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.100377    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-service-ca" (OuterVolumeSpecName: "service-ca") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:32:07.100432 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.100391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-console-config" (OuterVolumeSpecName: "console-config") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:32:07.102020 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.101988    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:32:07.102119 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.102024    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:32:07.102119 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.102052    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaf7193-9d0f-4103-8c23-8be8421660ca-kube-api-access-cv9vz" (OuterVolumeSpecName: "kube-api-access-cv9vz") pod "caaf7193-9d0f-4103-8c23-8be8421660ca" (UID: "caaf7193-9d0f-4103-8c23-8be8421660ca"). InnerVolumeSpecName "kube-api-access-cv9vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:32:07.200900 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.200850    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.200900 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.200891    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-service-ca\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.200900 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.200906    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-console-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.201200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.200920    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-trusted-ca-bundle\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.201200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.200973    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv9vz\" (UniqueName: \"kubernetes.io/projected/caaf7193-9d0f-4103-8c23-8be8421660ca-kube-api-access-cv9vz\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.201200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.200991    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/caaf7193-9d0f-4103-8c23-8be8421660ca-console-oauth-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.201200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.201006    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/caaf7193-9d0f-4103-8c23-8be8421660ca-oauth-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:07.745513 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.745476    2562 generic.go:356] "Generic (PLEG): container finished" podID="caaf7193-9d0f-4103-8c23-8be8421660ca" containerID="66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20" exitCode=0
Jul 29 10:32:07.745979 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.745535    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c94bd674-zvnw4" event={"ID":"caaf7193-9d0f-4103-8c23-8be8421660ca","Type":"ContainerDied","Data":"66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20"}
Jul 29 10:32:07.745979 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.745580    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c94bd674-zvnw4" event={"ID":"caaf7193-9d0f-4103-8c23-8be8421660ca","Type":"ContainerDied","Data":"b70a02878068d7309989557c1b15278505a3369040bb626a268a2e58e63a16c4"}
Jul 29 10:32:07.745979 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.745600    2562 scope.go:117] "RemoveContainer" containerID="66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20"
Jul 29 10:32:07.745979 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.745536    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c94bd674-zvnw4"
Jul 29 10:32:07.753995 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.753971    2562 scope.go:117] "RemoveContainer" containerID="66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20"
Jul 29 10:32:07.754295 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:07.754274    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20\": container with ID starting with 66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20 not found: ID does not exist" containerID="66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20"
Jul 29 10:32:07.754387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.754304    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20"} err="failed to get container status \"66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20\": rpc error: code = NotFound desc = could not find container \"66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20\": container with ID starting with 66f7e9b7acd026e11e585e537885e5edc2f439d753a2f113d653ab2eabf11e20 not found: ID does not exist"
Jul 29 10:32:07.764376 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.764347    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c94bd674-zvnw4"]
Jul 29 10:32:07.769894 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:07.769827    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78c94bd674-zvnw4"]
Jul 29 10:32:09.506411 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:09.506376    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caaf7193-9d0f-4103-8c23-8be8421660ca" path="/var/lib/kubelet/pods/caaf7193-9d0f-4103-8c23-8be8421660ca/volumes"
Jul 29 10:32:12.885204 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:12.885166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:12.988168 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:12.988131    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:13.458893 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:13.458857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:13.560687 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:13.560655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:13.662745 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:13.662706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:14.394366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.394313    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"]
Jul 29 10:32:14.394818 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.394801    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caaf7193-9d0f-4103-8c23-8be8421660ca" containerName="console"
Jul 29 10:32:14.394818 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.394818    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaf7193-9d0f-4103-8c23-8be8421660ca" containerName="console"
Jul 29 10:32:14.394918 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.394900    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="caaf7193-9d0f-4103-8c23-8be8421660ca" containerName="console"
Jul 29 10:32:14.399477 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.399457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.402739 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.402707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 29 10:32:14.402739 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.402726    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-82wjf\""
Jul 29 10:32:14.403932 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.403911    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 29 10:32:14.409187 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.409158    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"]
Jul 29 10:32:14.573780 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.573740    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.573982 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.573820    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnhgx\" (UniqueName: \"kubernetes.io/projected/2984b77e-3d7a-4f16-8743-5d2b523c728b-kube-api-access-wnhgx\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.573982 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.573866    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.674401 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.674263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.674562 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.674401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnhgx\" (UniqueName: \"kubernetes.io/projected/2984b77e-3d7a-4f16-8743-5d2b523c728b-kube-api-access-wnhgx\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.674562 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.674450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.674762 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.674739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.674798 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.674743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.684284 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.684249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnhgx\" (UniqueName: \"kubernetes.io/projected/2984b77e-3d7a-4f16-8743-5d2b523c728b-kube-api-access-wnhgx\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.709731 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.709692    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:14.855136 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:14.855112    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"]
Jul 29 10:32:14.857421 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:32:14.857392    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2984b77e_3d7a_4f16_8743_5d2b523c728b.slice/crio-cbb2304b45081524296a634fcecccff9cd0dba60297c532194c21fe47a958cfa WatchSource:0}: Error finding container cbb2304b45081524296a634fcecccff9cd0dba60297c532194c21fe47a958cfa: Status 404 returned error can't find the container with id cbb2304b45081524296a634fcecccff9cd0dba60297c532194c21fe47a958cfa
Jul 29 10:32:15.785875 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:15.785832    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq" event={"ID":"2984b77e-3d7a-4f16-8743-5d2b523c728b","Type":"ContainerStarted","Data":"cbb2304b45081524296a634fcecccff9cd0dba60297c532194c21fe47a958cfa"}
Jul 29 10:32:18.289341 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:18.289276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:18.391171 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:18.391132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:21.813976 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:21.813935    2562 generic.go:356] "Generic (PLEG): container finished" podID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerID="979cb2b979767b02441606bf00b10c7fe459ef79332eb0459de3acb188a78cbb" exitCode=0
Jul 29 10:32:21.814439 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:21.814025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq" event={"ID":"2984b77e-3d7a-4f16-8743-5d2b523c728b","Type":"ContainerDied","Data":"979cb2b979767b02441606bf00b10c7fe459ef79332eb0459de3acb188a78cbb"}
Jul 29 10:32:32.861228 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:32.861190    2562 generic.go:356] "Generic (PLEG): container finished" podID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerID="5fd3c57656dc9bf3da48e43b2ab426bc07ce83de0eb309535bd8c6137a5a5aaf" exitCode=0
Jul 29 10:32:32.861633 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:32.861266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq" event={"ID":"2984b77e-3d7a-4f16-8743-5d2b523c728b","Type":"ContainerDied","Data":"5fd3c57656dc9bf3da48e43b2ab426bc07ce83de0eb309535bd8c6137a5a5aaf"}
Jul 29 10:32:33.297150 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:33.297108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:39.899705 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:39.899671    2562 generic.go:356] "Generic (PLEG): container finished" podID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerID="fe52193b4b1426d244118e8c2ce2ea3abe11b7ea59242a84f19288c71b2643ff" exitCode=0
Jul 29 10:32:39.900154 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:39.899751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq" event={"ID":"2984b77e-3d7a-4f16-8743-5d2b523c728b","Type":"ContainerDied","Data":"fe52193b4b1426d244118e8c2ce2ea3abe11b7ea59242a84f19288c71b2643ff"}
Jul 29 10:32:41.029725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.029699    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:41.107992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.107954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-bundle\") pod \"2984b77e-3d7a-4f16-8743-5d2b523c728b\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") "
Jul 29 10:32:41.108184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.108019    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-util\") pod \"2984b77e-3d7a-4f16-8743-5d2b523c728b\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") "
Jul 29 10:32:41.108184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.108091    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnhgx\" (UniqueName: \"kubernetes.io/projected/2984b77e-3d7a-4f16-8743-5d2b523c728b-kube-api-access-wnhgx\") pod \"2984b77e-3d7a-4f16-8743-5d2b523c728b\" (UID: \"2984b77e-3d7a-4f16-8743-5d2b523c728b\") "
Jul 29 10:32:41.108667 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.108639    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-bundle" (OuterVolumeSpecName: "bundle") pod "2984b77e-3d7a-4f16-8743-5d2b523c728b" (UID: "2984b77e-3d7a-4f16-8743-5d2b523c728b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:32:41.110275 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.110250    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2984b77e-3d7a-4f16-8743-5d2b523c728b-kube-api-access-wnhgx" (OuterVolumeSpecName: "kube-api-access-wnhgx") pod "2984b77e-3d7a-4f16-8743-5d2b523c728b" (UID: "2984b77e-3d7a-4f16-8743-5d2b523c728b"). InnerVolumeSpecName "kube-api-access-wnhgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:32:41.112467 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.112443    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-util" (OuterVolumeSpecName: "util") pod "2984b77e-3d7a-4f16-8743-5d2b523c728b" (UID: "2984b77e-3d7a-4f16-8743-5d2b523c728b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 10:32:41.209626 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.209528    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnhgx\" (UniqueName: \"kubernetes.io/projected/2984b77e-3d7a-4f16-8743-5d2b523c728b-kube-api-access-wnhgx\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:41.209626 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.209571    2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-bundle\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:41.209626 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.209583    2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2984b77e-3d7a-4f16-8743-5d2b523c728b-util\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:32:41.909724 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.909635    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq"
Jul 29 10:32:41.909724 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.909629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l2hjq" event={"ID":"2984b77e-3d7a-4f16-8743-5d2b523c728b","Type":"ContainerDied","Data":"cbb2304b45081524296a634fcecccff9cd0dba60297c532194c21fe47a958cfa"}
Jul 29 10:32:41.909948 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:41.909751    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbb2304b45081524296a634fcecccff9cd0dba60297c532194c21fe47a958cfa"
Jul 29 10:32:45.774741 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:45.774708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:45.874048 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:45.874012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:46.906204 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906166    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"]
Jul 29 10:32:46.906777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906678    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="extract"
Jul 29 10:32:46.906777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906699    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="extract"
Jul 29 10:32:46.906777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906738    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="util"
Jul 29 10:32:46.906777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906747    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="util"
Jul 29 10:32:46.906777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906763    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="pull"
Jul 29 10:32:46.906777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906771    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="pull"
Jul 29 10:32:46.907063 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.906855    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2984b77e-3d7a-4f16-8743-5d2b523c728b" containerName="extract"
Jul 29 10:32:46.910165 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.910144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:46.913294 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.913273    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 29 10:32:46.913484 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.913427    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 29 10:32:46.913543 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.913495    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 29 10:32:46.913589 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.913570    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-zf7nj\""
Jul 29 10:32:46.923046 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:46.923009    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"]
Jul 29 10:32:47.066556 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.066518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vwsz\" (UniqueName: \"kubernetes.io/projected/6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f-kube-api-access-2vwsz\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9\" (UID: \"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.066752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.066604    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9\" (UID: \"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.168022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.167912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9\" (UID: \"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.168022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.167968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vwsz\" (UniqueName: \"kubernetes.io/projected/6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f-kube-api-access-2vwsz\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9\" (UID: \"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.170478 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.170450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9\" (UID: \"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.180151 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.180122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vwsz\" (UniqueName: \"kubernetes.io/projected/6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f-kube-api-access-2vwsz\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9\" (UID: \"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.221130 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.221090    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:47.363868 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.363839    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"]
Jul 29 10:32:47.366773 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:32:47.366743    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd06057_0e5b_4f28_990f_ae2c4a6f0b6f.slice/crio-728abd7da27c9644bc23482322bff0e0e342bc9bae2a6f1109f18927ee94fb9b WatchSource:0}: Error finding container 728abd7da27c9644bc23482322bff0e0e342bc9bae2a6f1109f18927ee94fb9b: Status 404 returned error can't find the container with id 728abd7da27c9644bc23482322bff0e0e342bc9bae2a6f1109f18927ee94fb9b
Jul 29 10:32:47.936007 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:47.935920    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9" event={"ID":"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f","Type":"ContainerStarted","Data":"728abd7da27c9644bc23482322bff0e0e342bc9bae2a6f1109f18927ee94fb9b"}
Jul 29 10:32:51.395746 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.395710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:51.670009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.669914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:51.673364 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.673308    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-ghdfh"]
Jul 29 10:32:51.676934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.676910    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.680119 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.680095    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 29 10:32:51.680638 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.680620    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-v66db\""
Jul 29 10:32:51.680887 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.680866    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 29 10:32:51.688398 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.688373    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-ghdfh"]
Jul 29 10:32:51.813625 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.813580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gfth\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-kube-api-access-8gfth\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.813815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.813718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-certificates\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.813815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.813761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-cabundle0\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.874513 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.874484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:51.896071 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.896033    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"]
Jul 29 10:32:51.899437 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.899417    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:51.903262 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.903230    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 29 10:32:51.914315 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.914277    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"]
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.914285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-certificates\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.914400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-cabundle0\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:51.914401    2562 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:51.914426    2562 secret.go:281] references non-existent secret key: ca.crt
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:51.914439    2562 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.914438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gfth\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-kube-api-access-8gfth\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:51.914455    2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-ghdfh: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 29 10:32:51.914512 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:51.914510    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-certificates podName:cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a nodeName:}" failed. No retries permitted until 2026-07-29 10:32:52.414491303 +0000 UTC m=+285.403340741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-certificates") pod "keda-operator-77f648d694-ghdfh" (UID: "cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 29 10:32:51.915285 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.915263    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-cabundle0\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.931048 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.930970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gfth\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-kube-api-access-8gfth\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:51.959057 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.959018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9" event={"ID":"6bd06057-0e5b-4f28-990f-ae2c4a6f0b6f","Type":"ContainerStarted","Data":"9d572d223645acc73bb7fff5a65116a85c88d0bb8675aa7c410ab2f6563eb769"}
Jul 29 10:32:51.959267 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.959138    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:32:51.984239 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:51.984183    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9" podStartSLOduration=2.436798593 podStartE2EDuration="5.984164884s" totalImagesPullingTime="3.547366291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:32:51.982984214 +0000 UTC m=+284.971833661" watchObservedRunningTime="2026-07-29 10:32:51.984164884 +0000 UTC m=+284.973014351"
Jul 29 10:32:52.015098 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.015054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-certificates\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.015291 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.015169    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6vk\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-kube-api-access-jx6vk\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.015291 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.015209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/55174e05-f91c-4381-894e-fcd2ffd9ea0b-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.018160 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.018125    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.116653 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.116608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-certificates\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.116863 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.116703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6vk\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-kube-api-access-jx6vk\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.116863 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.116722    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/55174e05-f91c-4381-894e-fcd2ffd9ea0b-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.116863 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.116768    2562 secret.go:281] references non-existent secret key: tls.crt
Jul 29 10:32:52.116863 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.116818    2562 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt
Jul 29 10:32:52.116863 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.116843    2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l: references non-existent secret key: tls.crt
Jul 29 10:32:52.117108 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.116905    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-certificates podName:55174e05-f91c-4381-894e-fcd2ffd9ea0b nodeName:}" failed. No retries permitted until 2026-07-29 10:32:52.616885063 +0000 UTC m=+285.605734498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-certificates") pod "keda-metrics-apiserver-55557df8c-hjw5l" (UID: "55174e05-f91c-4381-894e-fcd2ffd9ea0b") : references non-existent secret key: tls.crt
Jul 29 10:32:52.117195 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.117167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/55174e05-f91c-4381-894e-fcd2ffd9ea0b-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.125787 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.125748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6vk\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-kube-api-access-jx6vk\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.272954 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.272914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.360853 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.360817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.364113 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.364082    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-fhx59"]
Jul 29 10:32:52.367870 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.367847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:52.370781 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.370757    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 29 10:32:52.379387 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.379357    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-fhx59"]
Jul 29 10:32:52.420349 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.420291    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-certificates\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:52.422764 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.422731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a-certificates\") pod \"keda-operator-77f648d694-ghdfh\" (UID: \"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a\") " pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:52.436924 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.436887    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.521341 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.521291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h8pl\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-kube-api-access-9h8pl\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:52.521544 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.521447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-certificates\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:52.588675 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.588580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:32:52.622804 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.622769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-certificates\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.623003 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.622823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-certificates\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:52.623003 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.622940    2562 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 29 10:32:52.623003 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.622964    2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-fhx59: secret "keda-admission-webhooks-certs" not found
Jul 29 10:32:52.623162 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:32:52.623025    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-certificates podName:62d26a44-90f5-4e80-85d1-72ae7bb5a6af nodeName:}" failed. No retries permitted until 2026-07-29 10:32:53.123005283 +0000 UTC m=+286.111854733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-certificates") pod "keda-admission-56f9798bb4-fhx59" (UID: "62d26a44-90f5-4e80-85d1-72ae7bb5a6af") : secret "keda-admission-webhooks-certs" not found
Jul 29 10:32:52.623162 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.622888    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h8pl\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-kube-api-access-9h8pl\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:52.625438 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.625403    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/55174e05-f91c-4381-894e-fcd2ffd9ea0b-certificates\") pod \"keda-metrics-apiserver-55557df8c-hjw5l\" (UID: \"55174e05-f91c-4381-894e-fcd2ffd9ea0b\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.632634 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.632602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h8pl\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-kube-api-access-9h8pl\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:52.709117 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.709084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.742751 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.742721    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-ghdfh"]
Jul 29 10:32:52.745226 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:32:52.745199    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf8b3f9_edcd_4f01_abe5_681c0ae64b8a.slice/crio-085eb2e94dad7b8d179f6078faa4ddf05a5f592de1b9ef5666498acc9b6da52a WatchSource:0}: Error finding container 085eb2e94dad7b8d179f6078faa4ddf05a5f592de1b9ef5666498acc9b6da52a: Status 404 returned error can't find the container with id 085eb2e94dad7b8d179f6078faa4ddf05a5f592de1b9ef5666498acc9b6da52a
Jul 29 10:32:52.801040 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.801009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.809676 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.809644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:52.897208 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.897165    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:52.955089 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.955065    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"]
Jul 29 10:32:52.957760 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:32:52.957730    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55174e05_f91c_4381_894e_fcd2ffd9ea0b.slice/crio-4df90ae29e50cf5d2ac1e39fdb5b3b00c21e181c19c7c8645c6a443ea8982fe4 WatchSource:0}: Error finding container 4df90ae29e50cf5d2ac1e39fdb5b3b00c21e181c19c7c8645c6a443ea8982fe4: Status 404 returned error can't find the container with id 4df90ae29e50cf5d2ac1e39fdb5b3b00c21e181c19c7c8645c6a443ea8982fe4
Jul 29 10:32:52.966082 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.966054    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-ghdfh" event={"ID":"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a","Type":"ContainerStarted","Data":"085eb2e94dad7b8d179f6078faa4ddf05a5f592de1b9ef5666498acc9b6da52a"}
Jul 29 10:32:52.967209 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.967183    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l" event={"ID":"55174e05-f91c-4381-894e-fcd2ffd9ea0b","Type":"ContainerStarted","Data":"4df90ae29e50cf5d2ac1e39fdb5b3b00c21e181c19c7c8645c6a443ea8982fe4"}
Jul 29 10:32:52.996769 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:52.996732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:32:53.128776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:53.128677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-certificates\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:53.131241 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:53.131220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/62d26a44-90f5-4e80-85d1-72ae7bb5a6af-certificates\") pod \"keda-admission-56f9798bb4-fhx59\" (UID: \"62d26a44-90f5-4e80-85d1-72ae7bb5a6af\") " pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:53.280919 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:53.280875    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:53.418268 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:53.418226    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-fhx59"]
Jul 29 10:32:53.420508 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:32:53.420460    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d26a44_90f5_4e80_85d1_72ae7bb5a6af.slice/crio-4af1043ac299907bbb1949f0d94c1b5752a0dfa1b1959a7cbe758cb444d433c4 WatchSource:0}: Error finding container 4af1043ac299907bbb1949f0d94c1b5752a0dfa1b1959a7cbe758cb444d433c4: Status 404 returned error can't find the container with id 4af1043ac299907bbb1949f0d94c1b5752a0dfa1b1959a7cbe758cb444d433c4
Jul 29 10:32:53.973250 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:53.973215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-fhx59" event={"ID":"62d26a44-90f5-4e80-85d1-72ae7bb5a6af","Type":"ContainerStarted","Data":"4af1043ac299907bbb1949f0d94c1b5752a0dfa1b1959a7cbe758cb444d433c4"}
Jul 29 10:32:54.980620 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:54.980581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-fhx59" event={"ID":"62d26a44-90f5-4e80-85d1-72ae7bb5a6af","Type":"ContainerStarted","Data":"c69fe4dc941b5b42188db41c0e0dd7befa12e667cddf4b1629c3ae50a50276e1"}
Jul 29 10:32:54.981097 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:54.980721    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:32:55.000158 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:55.000092    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-fhx59" podStartSLOduration=1.5197174310000001 podStartE2EDuration="3.000070443s" totalImagesPullingTime="1.480353012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:32:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:32:54.999022118 +0000 UTC m=+287.987871574" watchObservedRunningTime="2026-07-29 10:32:55.000070443 +0000 UTC m=+287.988919923"
Jul 29 10:32:56.993476 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:56.993441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l" event={"ID":"55174e05-f91c-4381-894e-fcd2ffd9ea0b","Type":"ContainerStarted","Data":"f58bd6008ba0111132883b3f540fce9003a7b46f4a81b73a2e5faf0845465621"}
Jul 29 10:32:56.993925 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:56.993580    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:32:57.012437 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:32:57.012382    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l" podStartSLOduration=2.11804793 podStartE2EDuration="6.01236449s" totalImagesPullingTime="3.89431656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:32:57.011469997 +0000 UTC m=+290.000319453" watchObservedRunningTime="2026-07-29 10:32:57.01236449 +0000 UTC m=+290.001213946"
Jul 29 10:33:02.513068 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:02.513017    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:33:05.035223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:05.035186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-ghdfh" event={"ID":"cdf8b3f9-edcd-4f01-abe5-681c0ae64b8a","Type":"ContainerStarted","Data":"432344c8e8afad127c41584f38a7a69c6e4b086e80051168c41d2f8812b81eea"}
Jul 29 10:33:05.035673 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:05.035350    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:33:05.053570 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:05.053504    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-ghdfh" podStartSLOduration=2.250104964 podStartE2EDuration="14.053485519s" totalImagesPullingTime="11.803380555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:33:05.051673651 +0000 UTC m=+298.040523107" watchObservedRunningTime="2026-07-29 10:33:05.053485519 +0000 UTC m=+298.042334974"
Jul 29 10:33:07.455534 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:07.455504    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:33:07.457969 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:07.457946    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:33:07.466955 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:07.466930    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 10:33:08.003253 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:08.003219    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-hjw5l"
Jul 29 10:33:12.970394 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:12.970360    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-jwwt9"
Jul 29 10:33:15.988877 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:15.988843    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-fhx59"
Jul 29 10:33:25.938081 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:25.938044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:33:26.041084 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:26.041050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:33:26.041716 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:26.041694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-ghdfh"
Jul 29 10:33:56.508773 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:56.508739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:33:58.053513 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:33:58.053472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:09.672184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.672081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:09.725901 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.725866    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-mkwbr"]
Jul 29 10:34:09.729453 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.729425    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.732149 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.732120    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 29 10:34:09.732302 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.732152    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 29 10:34:09.732302 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.732280    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 29 10:34:09.733526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.733511    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-cvtlp\""
Jul 29 10:34:09.737841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.737817    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-szcn7"]
Jul 29 10:34:09.741718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.741693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.743496 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.743469    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-mkwbr"]
Jul 29 10:34:09.744583 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.744558    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 29 10:34:09.744710 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.744610    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-564v9\""
Jul 29 10:34:09.753902 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.753868    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-szcn7"]
Jul 29 10:34:09.764104 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.764069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:09.804134 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.804094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f9752ff-f6c8-4235-880c-82115c1ecf21-cert\") pod \"kserve-controller-manager-65f566c74b-mkwbr\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") " pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.804348 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.804149    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60153e75-8158-4de3-903c-dcd74bcc3013-cert\") pod \"llmisvc-controller-manager-84fc959947-szcn7\" (UID: \"60153e75-8158-4de3-903c-dcd74bcc3013\") " pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.804348 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.804219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t59t5\" (UniqueName: \"kubernetes.io/projected/0f9752ff-f6c8-4235-880c-82115c1ecf21-kube-api-access-t59t5\") pod \"kserve-controller-manager-65f566c74b-mkwbr\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") " pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.804348 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.804241    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpj9p\" (UniqueName: \"kubernetes.io/projected/60153e75-8158-4de3-903c-dcd74bcc3013-kube-api-access-lpj9p\") pod \"llmisvc-controller-manager-84fc959947-szcn7\" (UID: \"60153e75-8158-4de3-903c-dcd74bcc3013\") " pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.839014 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.838979    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:09.905685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.905649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60153e75-8158-4de3-903c-dcd74bcc3013-cert\") pod \"llmisvc-controller-manager-84fc959947-szcn7\" (UID: \"60153e75-8158-4de3-903c-dcd74bcc3013\") " pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.905871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.905726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t59t5\" (UniqueName: \"kubernetes.io/projected/0f9752ff-f6c8-4235-880c-82115c1ecf21-kube-api-access-t59t5\") pod \"kserve-controller-manager-65f566c74b-mkwbr\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") " pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.905871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.905753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpj9p\" (UniqueName: \"kubernetes.io/projected/60153e75-8158-4de3-903c-dcd74bcc3013-kube-api-access-lpj9p\") pod \"llmisvc-controller-manager-84fc959947-szcn7\" (UID: \"60153e75-8158-4de3-903c-dcd74bcc3013\") " pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.905871 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.905795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f9752ff-f6c8-4235-880c-82115c1ecf21-cert\") pod \"kserve-controller-manager-65f566c74b-mkwbr\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") " pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.908160 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.908124    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f9752ff-f6c8-4235-880c-82115c1ecf21-cert\") pod \"kserve-controller-manager-65f566c74b-mkwbr\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") " pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.908160 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.908145    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60153e75-8158-4de3-903c-dcd74bcc3013-cert\") pod \"llmisvc-controller-manager-84fc959947-szcn7\" (UID: \"60153e75-8158-4de3-903c-dcd74bcc3013\") " pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.914281 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.914254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t59t5\" (UniqueName: \"kubernetes.io/projected/0f9752ff-f6c8-4235-880c-82115c1ecf21-kube-api-access-t59t5\") pod \"kserve-controller-manager-65f566c74b-mkwbr\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") " pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:09.914281 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.914272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpj9p\" (UniqueName: \"kubernetes.io/projected/60153e75-8158-4de3-903c-dcd74bcc3013-kube-api-access-lpj9p\") pod \"llmisvc-controller-manager-84fc959947-szcn7\" (UID: \"60153e75-8158-4de3-903c-dcd74bcc3013\") " pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:09.936989 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:09.936863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:10.041932 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.041894    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:10.054243 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.054217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:10.197530 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.195125    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-mkwbr"]
Jul 29 10:34:10.202281 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.202161    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 10:34:10.222999 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.222975    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-szcn7"]
Jul 29 10:34:10.225604 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:34:10.225572    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod60153e75_8158_4de3_903c_dcd74bcc3013.slice/crio-434da723930b4fb272a5a1777073fcb821e4e40cbf8426d3c89a2f7ac8831528 WatchSource:0}: Error finding container 434da723930b4fb272a5a1777073fcb821e4e40cbf8426d3c89a2f7ac8831528: Status 404 returned error can't find the container with id 434da723930b4fb272a5a1777073fcb821e4e40cbf8426d3c89a2f7ac8831528
Jul 29 10:34:10.280114 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.280060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:10.340480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.340437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7" event={"ID":"60153e75-8158-4de3-903c-dcd74bcc3013","Type":"ContainerStarted","Data":"434da723930b4fb272a5a1777073fcb821e4e40cbf8426d3c89a2f7ac8831528"}
Jul 29 10:34:10.341536 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.341510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr" event={"ID":"0f9752ff-f6c8-4235-880c-82115c1ecf21","Type":"ContainerStarted","Data":"444d513211fed2a36a2c3621a54f9da97e90df2907d19edc4e6a8cea9bb0ef03"}
Jul 29 10:34:10.382095 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.382064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:10.444855 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:10.444824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:14.085112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.085018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:14.366808 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.366713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7" event={"ID":"60153e75-8158-4de3-903c-dcd74bcc3013","Type":"ContainerStarted","Data":"8af576649a8aabb85fe396098204c4057c31dbcd42ccf196e38a4cae6fbf182f"}
Jul 29 10:34:14.367006 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.366828    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:14.368271 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.368249    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr" event={"ID":"0f9752ff-f6c8-4235-880c-82115c1ecf21","Type":"ContainerStarted","Data":"28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3"}
Jul 29 10:34:14.368377 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.368360    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:14.384732 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.384680    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7" podStartSLOduration=1.859046281 podStartE2EDuration="5.38466212s" totalImagesPullingTime="3.525615839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:34:14.383197534 +0000 UTC m=+367.372046992" watchObservedRunningTime="2026-07-29 10:34:14.38466212 +0000 UTC m=+367.373511576"
Jul 29 10:34:14.400831 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:14.400777    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr" podStartSLOduration=1.850711198 podStartE2EDuration="5.400759152s" totalImagesPullingTime="3.550047954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:34:14.40011763 +0000 UTC m=+367.388967086" watchObservedRunningTime="2026-07-29 10:34:14.400759152 +0000 UTC m=+367.389608644"
Jul 29 10:34:28.966707 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:28.966667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:34:45.381067 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:45.381031    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-84fc959947-szcn7"
Jul 29 10:34:45.384193 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:45.384164    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:53.527001 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.526960    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-mkwbr"]
Jul 29 10:34:53.527530 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.527265    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr" podUID="0f9752ff-f6c8-4235-880c-82115c1ecf21" containerName="manager" containerID="cri-o://28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3" gracePeriod=10
Jul 29 10:34:53.551275 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.551243    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-m2xzd"]
Jul 29 10:34:53.554580 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.554555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.563436 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.563397    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-m2xzd"]
Jul 29 10:34:53.603398 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.603356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4a91a83-390d-45ea-b57f-4955428dc0a0-cert\") pod \"kserve-controller-manager-65f566c74b-m2xzd\" (UID: \"a4a91a83-390d-45ea-b57f-4955428dc0a0\") " pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.603613 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.603488    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfz76\" (UniqueName: \"kubernetes.io/projected/a4a91a83-390d-45ea-b57f-4955428dc0a0-kube-api-access-gfz76\") pod \"kserve-controller-manager-65f566c74b-m2xzd\" (UID: \"a4a91a83-390d-45ea-b57f-4955428dc0a0\") " pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.704906 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.704872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfz76\" (UniqueName: \"kubernetes.io/projected/a4a91a83-390d-45ea-b57f-4955428dc0a0-kube-api-access-gfz76\") pod \"kserve-controller-manager-65f566c74b-m2xzd\" (UID: \"a4a91a83-390d-45ea-b57f-4955428dc0a0\") " pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.705101 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.704988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4a91a83-390d-45ea-b57f-4955428dc0a0-cert\") pod \"kserve-controller-manager-65f566c74b-m2xzd\" (UID: \"a4a91a83-390d-45ea-b57f-4955428dc0a0\") " pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.707500 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.707472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a4a91a83-390d-45ea-b57f-4955428dc0a0-cert\") pod \"kserve-controller-manager-65f566c74b-m2xzd\" (UID: \"a4a91a83-390d-45ea-b57f-4955428dc0a0\") " pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.713908 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.713815    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfz76\" (UniqueName: \"kubernetes.io/projected/a4a91a83-390d-45ea-b57f-4955428dc0a0-kube-api-access-gfz76\") pod \"kserve-controller-manager-65f566c74b-m2xzd\" (UID: \"a4a91a83-390d-45ea-b57f-4955428dc0a0\") " pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:53.774916 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.774880    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:53.805445 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.805351    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f9752ff-f6c8-4235-880c-82115c1ecf21-cert\") pod \"0f9752ff-f6c8-4235-880c-82115c1ecf21\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") "
Jul 29 10:34:53.805445 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.805416    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t59t5\" (UniqueName: \"kubernetes.io/projected/0f9752ff-f6c8-4235-880c-82115c1ecf21-kube-api-access-t59t5\") pod \"0f9752ff-f6c8-4235-880c-82115c1ecf21\" (UID: \"0f9752ff-f6c8-4235-880c-82115c1ecf21\") "
Jul 29 10:34:53.807634 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.807604    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f9752ff-f6c8-4235-880c-82115c1ecf21-kube-api-access-t59t5" (OuterVolumeSpecName: "kube-api-access-t59t5") pod "0f9752ff-f6c8-4235-880c-82115c1ecf21" (UID: "0f9752ff-f6c8-4235-880c-82115c1ecf21"). InnerVolumeSpecName "kube-api-access-t59t5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:34:53.807634 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.807613    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f9752ff-f6c8-4235-880c-82115c1ecf21-cert" (OuterVolumeSpecName: "cert") pod "0f9752ff-f6c8-4235-880c-82115c1ecf21" (UID: "0f9752ff-f6c8-4235-880c-82115c1ecf21"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:34:53.906461 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.906424    2562 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0f9752ff-f6c8-4235-880c-82115c1ecf21-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:34:53.906461 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.906468    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t59t5\" (UniqueName: \"kubernetes.io/projected/0f9752ff-f6c8-4235-880c-82115c1ecf21-kube-api-access-t59t5\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:34:53.918085 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:53.918049    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:54.054152 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.053957    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-m2xzd"]
Jul 29 10:34:54.057085 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:34:54.057024    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a91a83_390d_45ea_b57f_4955428dc0a0.slice/crio-e236ed45539edfe1644960d4674bf7741439e7616d9fdf6a38a954ab61a2a8fa WatchSource:0}: Error finding container e236ed45539edfe1644960d4674bf7741439e7616d9fdf6a38a954ab61a2a8fa: Status 404 returned error can't find the container with id e236ed45539edfe1644960d4674bf7741439e7616d9fdf6a38a954ab61a2a8fa
Jul 29 10:34:54.558062 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.558021    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd" event={"ID":"a4a91a83-390d-45ea-b57f-4955428dc0a0","Type":"ContainerStarted","Data":"4a18bb1a3a481100c02a7fd1c5b00235a3fd8a425ad0af33c462df3f653b0090"}
Jul 29 10:34:54.558558 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.558098    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:34:54.558558 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.558125    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd" event={"ID":"a4a91a83-390d-45ea-b57f-4955428dc0a0","Type":"ContainerStarted","Data":"e236ed45539edfe1644960d4674bf7741439e7616d9fdf6a38a954ab61a2a8fa"}
Jul 29 10:34:54.559408 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.559378    2562 generic.go:356] "Generic (PLEG): container finished" podID="0f9752ff-f6c8-4235-880c-82115c1ecf21" containerID="28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3" exitCode=0
Jul 29 10:34:54.559484 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.559411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr"
Jul 29 10:34:54.559484 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.559452    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr" event={"ID":"0f9752ff-f6c8-4235-880c-82115c1ecf21","Type":"ContainerDied","Data":"28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3"}
Jul 29 10:34:54.559550 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.559507    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-65f566c74b-mkwbr" event={"ID":"0f9752ff-f6c8-4235-880c-82115c1ecf21","Type":"ContainerDied","Data":"444d513211fed2a36a2c3621a54f9da97e90df2907d19edc4e6a8cea9bb0ef03"}
Jul 29 10:34:54.559550 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.559535    2562 scope.go:117] "RemoveContainer" containerID="28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3"
Jul 29 10:34:54.569064 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.569044    2562 scope.go:117] "RemoveContainer" containerID="28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3"
Jul 29 10:34:54.569385 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:34:54.569363    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3\": container with ID starting with 28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3 not found: ID does not exist" containerID="28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3"
Jul 29 10:34:54.569458 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.569396    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3"} err="failed to get container status \"28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3\": rpc error: code = NotFound desc = could not find container \"28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3\": container with ID starting with 28f88fdbd7faba8b35a7e5c8ea3c075d0c1eff4285b26f1a19c83a45838549b3 not found: ID does not exist"
Jul 29 10:34:54.578442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.578388    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd" podStartSLOduration=1.140790917 podStartE2EDuration="1.578369223s" totalImagesPullingTime="437.578306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:34:54.576782414 +0000 UTC m=+407.565631872" watchObservedRunningTime="2026-07-29 10:34:54.578369223 +0000 UTC m=+407.567218679"
Jul 29 10:34:54.595179 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.595142    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-mkwbr"]
Jul 29 10:34:54.599090 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:54.599059    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-65f566c74b-mkwbr"]
Jul 29 10:34:55.507229 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:34:55.507196    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f9752ff-f6c8-4235-880c-82115c1ecf21" path="/var/lib/kubelet/pods/0f9752ff-f6c8-4235-880c-82115c1ecf21/volumes"
Jul 29 10:35:16.444514 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.444480    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8555ffc549-whvmc"]
Jul 29 10:35:16.445051 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.445030    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f9752ff-f6c8-4235-880c-82115c1ecf21" containerName="manager"
Jul 29 10:35:16.445129 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.445054    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f9752ff-f6c8-4235-880c-82115c1ecf21" containerName="manager"
Jul 29 10:35:16.445184 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.445141    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f9752ff-f6c8-4235-880c-82115c1ecf21" containerName="manager"
Jul 29 10:35:16.449537 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.449502    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.461222 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.461148    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8555ffc549-whvmc"]
Jul 29 10:35:16.519136 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.519075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:16.604090 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604018    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-oauth-serving-cert\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.604090 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3121acfc-b574-4032-8b77-47dff9312eb8-console-oauth-config\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.604090 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:16.604363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-trusted-ca-bundle\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.604363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdp2v\" (UniqueName: \"kubernetes.io/projected/3121acfc-b574-4032-8b77-47dff9312eb8-kube-api-access-xdp2v\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.604363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604208    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3121acfc-b574-4032-8b77-47dff9312eb8-console-serving-cert\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.604363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-console-config\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.604363 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.604275    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-service-ca\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705086 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.704980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-service-ca\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705086 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-oauth-serving-cert\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705090    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3121acfc-b574-4032-8b77-47dff9312eb8-console-oauth-config\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-trusted-ca-bundle\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdp2v\" (UniqueName: \"kubernetes.io/projected/3121acfc-b574-4032-8b77-47dff9312eb8-kube-api-access-xdp2v\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3121acfc-b574-4032-8b77-47dff9312eb8-console-serving-cert\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705371 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-console-config\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.705917 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.705893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-service-ca\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.706074 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.706055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-oauth-serving-cert\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.706204 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.706178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-trusted-ca-bundle\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.706265 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.706216    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3121acfc-b574-4032-8b77-47dff9312eb8-console-config\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.707793 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.707773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3121acfc-b574-4032-8b77-47dff9312eb8-console-oauth-config\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.708026 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.708005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3121acfc-b574-4032-8b77-47dff9312eb8-console-serving-cert\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.714977 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.714950    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdp2v\" (UniqueName: \"kubernetes.io/projected/3121acfc-b574-4032-8b77-47dff9312eb8-kube-api-access-xdp2v\") pod \"console-8555ffc549-whvmc\" (UID: \"3121acfc-b574-4032-8b77-47dff9312eb8\") " pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.760177 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.760140    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:16.902237 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:16.902208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8555ffc549-whvmc"]
Jul 29 10:35:16.904424 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:35:16.904392    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3121acfc_b574_4032_8b77_47dff9312eb8.slice/crio-f5fcd871a5b08d02daa8a5910c0a95df75c7c79a382f30e310ea7f7339bb2f47 WatchSource:0}: Error finding container f5fcd871a5b08d02daa8a5910c0a95df75c7c79a382f30e310ea7f7339bb2f47: Status 404 returned error can't find the container with id f5fcd871a5b08d02daa8a5910c0a95df75c7c79a382f30e310ea7f7339bb2f47
Jul 29 10:35:17.671560 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:17.671520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8555ffc549-whvmc" event={"ID":"3121acfc-b574-4032-8b77-47dff9312eb8","Type":"ContainerStarted","Data":"de89ef52f71cde01dcdfb877cee15e4e72675a11295bf69eccd09950b2483c6e"}
Jul 29 10:35:17.671972 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:17.671589    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8555ffc549-whvmc" event={"ID":"3121acfc-b574-4032-8b77-47dff9312eb8","Type":"ContainerStarted","Data":"f5fcd871a5b08d02daa8a5910c0a95df75c7c79a382f30e310ea7f7339bb2f47"}
Jul 29 10:35:17.692587 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:17.692528    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8555ffc549-whvmc" podStartSLOduration=1.692511046 podStartE2EDuration="1.692511046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:35:17.691832118 +0000 UTC m=+430.680681576" watchObservedRunningTime="2026-07-29 10:35:17.692511046 +0000 UTC m=+430.681360501"
Jul 29 10:35:25.571168 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:25.571141    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-65f566c74b-m2xzd"
Jul 29 10:35:26.579421 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:26.579381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:26.663299 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:26.663263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:26.761188 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:26.761138    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:26.761406 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:26.761251    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:26.766364 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:26.766313    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:27.035578 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.035544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:27.258448 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.258412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:27.335289 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.335198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:27.722916 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.722819    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8555ffc549-whvmc"
Jul 29 10:35:27.775157 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.775120    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5869686cdc-lgz7r"]
Jul 29 10:35:27.887343 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.887293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:27.988791 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:27.988761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:28.092787 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:28.092753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:35:52.796210 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:52.796151    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5869686cdc-lgz7r" podUID="36cf0f71-989a-48f7-854c-c69c22e0dd0d" containerName="console" containerID="cri-o://b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53" gracePeriod=15
Jul 29 10:35:53.051075 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.050993    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:35:53.135110 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135074    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-trusted-ca-bundle\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135110 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135116    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-config\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135160    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-service-ca\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135192    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-oauth-serving-cert\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135236    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gwtc\" (UniqueName: \"kubernetes.io/projected/36cf0f71-989a-48f7-854c-c69c22e0dd0d-kube-api-access-7gwtc\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135277    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-oauth-config\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135366 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-serving-cert\") pod \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\" (UID: \"36cf0f71-989a-48f7-854c-c69c22e0dd0d\") "
Jul 29 10:35:53.135653 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135623    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-service-ca" (OuterVolumeSpecName: "service-ca") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:35:53.135718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135650    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:35:53.135718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135664    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-config" (OuterVolumeSpecName: "console-config") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:35:53.135718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.135632    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:35:53.137618 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.137580    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:35:53.137748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.137680    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36cf0f71-989a-48f7-854c-c69c22e0dd0d-kube-api-access-7gwtc" (OuterVolumeSpecName: "kube-api-access-7gwtc") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "kube-api-access-7gwtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:35:53.137748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.137691    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36cf0f71-989a-48f7-854c-c69c22e0dd0d" (UID: "36cf0f71-989a-48f7-854c-c69c22e0dd0d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:35:53.236487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236442    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-trusted-ca-bundle\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.236487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236475    2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.236487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236484    2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-service-ca\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.236487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236492    2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36cf0f71-989a-48f7-854c-c69c22e0dd0d-oauth-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.236752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236501    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gwtc\" (UniqueName: \"kubernetes.io/projected/36cf0f71-989a-48f7-854c-c69c22e0dd0d-kube-api-access-7gwtc\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.236752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236511    2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-oauth-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.236752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.236518    2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36cf0f71-989a-48f7-854c-c69c22e0dd0d-console-serving-cert\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:35:53.844203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.844096    2562 generic.go:356] "Generic (PLEG): container finished" podID="36cf0f71-989a-48f7-854c-c69c22e0dd0d" containerID="b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53" exitCode=0
Jul 29 10:35:53.844203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.844149    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5869686cdc-lgz7r"
Jul 29 10:35:53.844203 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.844180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5869686cdc-lgz7r" event={"ID":"36cf0f71-989a-48f7-854c-c69c22e0dd0d","Type":"ContainerDied","Data":"b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53"}
Jul 29 10:35:53.844718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.844246    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5869686cdc-lgz7r" event={"ID":"36cf0f71-989a-48f7-854c-c69c22e0dd0d","Type":"ContainerDied","Data":"62129157bcb675428f0e9e1b894b41c88ed04d17be4fa0dc468b6aeaee5b1d74"}
Jul 29 10:35:53.844718 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.844268    2562 scope.go:117] "RemoveContainer" containerID="b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53"
Jul 29 10:35:53.854615 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.854594    2562 scope.go:117] "RemoveContainer" containerID="b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53"
Jul 29 10:35:53.854925 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:35:53.854905    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53\": container with ID starting with b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53 not found: ID does not exist" containerID="b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53"
Jul 29 10:35:53.854993 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.854937    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53"} err="failed to get container status \"b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53\": rpc error: code = NotFound desc = could not find container \"b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53\": container with ID starting with b11d5a829c55657c0782288a8fb39b135daee493e5e92fe102ecc3c70f926b53 not found: ID does not exist"
Jul 29 10:35:53.861740 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.861708    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5869686cdc-lgz7r"]
Jul 29 10:35:53.866137 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:53.866104    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5869686cdc-lgz7r"]
Jul 29 10:35:55.508069 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:35:55.508034    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36cf0f71-989a-48f7-854c-c69c22e0dd0d" path="/var/lib/kubelet/pods/36cf0f71-989a-48f7-854c-c69c22e0dd0d/volumes"
Jul 29 10:37:02.874748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:02.874652    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:02.963738 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:02.963706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:14.649645 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:14.649604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:14.798818 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:14.798786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:14.903287 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:14.903212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:14.987800 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:14.987763    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:15.030189 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.030144    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"]
Jul 29 10:37:15.030647 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.030616    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36cf0f71-989a-48f7-854c-c69c22e0dd0d" containerName="console"
Jul 29 10:37:15.030647 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.030650    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="36cf0f71-989a-48f7-854c-c69c22e0dd0d" containerName="console"
Jul 29 10:37:15.030779 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.030739    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="36cf0f71-989a-48f7-854c-c69c22e0dd0d" containerName="console"
Jul 29 10:37:15.034044 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.034023    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.036949 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.036923    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 29 10:37:15.037083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.036970    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0182-kube-rbac-proxy-sar-config\""
Jul 29 10:37:15.037083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.036981    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0182-predictor-serving-cert\""
Jul 29 10:37:15.038224 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.038205    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 29 10:37:15.038224 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.038215    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fqxzr\""
Jul 29 10:37:15.044211 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.044179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"]
Jul 29 10:37:15.051306 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.051279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:15.070130 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.070094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.070367 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.070142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6887\" (UniqueName: \"kubernetes.io/projected/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-api-access-s6887\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.070367 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.070282    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.136761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.136728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:15.171105 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.171008    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.171105 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.171069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6887\" (UniqueName: \"kubernetes.io/projected/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-api-access-s6887\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.171339 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.171134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.171339 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:37:15.171260    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-f0182-predictor-serving-cert: secret "error-404-isvc-f0182-predictor-serving-cert" not found
Jul 29 10:37:15.171434 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:37:15.171365    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls podName:dec3d21b-ccca-41e5-8be9-56c970cf4d5a nodeName:}" failed. No retries permitted until 2026-07-29 10:37:15.671313641 +0000 UTC m=+548.660163081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls") pod "error-404-isvc-f0182-predictor-6ff7979689-6jg5k" (UID: "dec3d21b-ccca-41e5-8be9-56c970cf4d5a") : secret "error-404-isvc-f0182-predictor-serving-cert" not found
Jul 29 10:37:15.171928 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.171904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.180293 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.180255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6887\" (UniqueName: \"kubernetes.io/projected/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-api-access-s6887\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.318823 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.318791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:15.533229 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.533199    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:15.675477 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.675432    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.677846 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.677821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls\") pod \"error-404-isvc-f0182-predictor-6ff7979689-6jg5k\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") " pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:15.704482 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.704448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:37:15.946361 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:15.946242    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:16.084187 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:16.084158    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"]
Jul 29 10:37:16.086758 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:37:16.086728    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddec3d21b_ccca_41e5_8be9_56c970cf4d5a.slice/crio-9b70bbee75491b230e026573668b10bfc88489565e3b06d6e4668297c0e8e601 WatchSource:0}: Error finding container 9b70bbee75491b230e026573668b10bfc88489565e3b06d6e4668297c0e8e601: Status 404 returned error can't find the container with id 9b70bbee75491b230e026573668b10bfc88489565e3b06d6e4668297c0e8e601
Jul 29 10:37:16.237018 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:16.236978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" event={"ID":"dec3d21b-ccca-41e5-8be9-56c970cf4d5a","Type":"ContainerStarted","Data":"9b70bbee75491b230e026573668b10bfc88489565e3b06d6e4668297c0e8e601"}
Jul 29 10:37:28.324551 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:28.324484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" event={"ID":"dec3d21b-ccca-41e5-8be9-56c970cf4d5a","Type":"ContainerStarted","Data":"41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d"}
Jul 29 10:37:31.343775 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:31.343730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" event={"ID":"dec3d21b-ccca-41e5-8be9-56c970cf4d5a","Type":"ContainerStarted","Data":"9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59"}
Jul 29 10:37:31.344216 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:31.343854    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:31.364771 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:31.364720    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podStartSLOduration=2.213193522 podStartE2EDuration="16.364702639s" totalImagesPullingTime="14.151509117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:37:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:37:31.363049371 +0000 UTC m=+564.351898829" watchObservedRunningTime="2026-07-29 10:37:31.364702639 +0000 UTC m=+564.353552094"
Jul 29 10:37:32.348205 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:32.348174    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:32.349793 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:32.349755    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 29 10:37:33.353606 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:33.353567    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 29 10:37:38.359689 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:38.359659    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:37:38.360171 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:38.360081    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 29 10:37:48.360618 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:48.360574    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 29 10:37:58.360480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:58.360431    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 29 10:37:59.459455 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:37:59.459415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:07.488805 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:07.488768    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:38:07.490039 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:07.490016    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:38:08.361108 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:08.361064    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 29 10:38:18.361490 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:18.361451    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:38:34.641193 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:34.641046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:34.739660 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:34.739620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:44.818511 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:44.818469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:44.928360 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:44.928291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:44.958762 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:44.958727    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"]
Jul 29 10:38:44.959050 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:44.959020    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" containerID="cri-o://41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d" gracePeriod=30
Jul 29 10:38:44.959154 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:44.959064    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kube-rbac-proxy" containerID="cri-o://9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59" gracePeriod=30
Jul 29 10:38:44.997889 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:44.997851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:45.084700 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.084672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:45.094607 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.094571    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"]
Jul 29 10:38:45.098408 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.098381    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.101233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.101208    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0a743-predictor-serving-cert\""
Jul 29 10:38:45.101422 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.101215    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0a743-kube-rbac-proxy-sar-config\""
Jul 29 10:38:45.107231 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.107205    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"]
Jul 29 10:38:45.181798 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.181742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:45.197117 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.197074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwjkx\" (UniqueName: \"kubernetes.io/projected/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-api-access-vwjkx\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.197304 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.197131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.197304 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.197168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.298419 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.298379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.298641 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.298436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.298641 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.298523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwjkx\" (UniqueName: \"kubernetes.io/projected/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-api-access-vwjkx\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.298641 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:38:45.298607    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-0a743-predictor-serving-cert: secret "error-404-isvc-0a743-predictor-serving-cert" not found
Jul 29 10:38:45.298807 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:38:45.298679    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls podName:03c31e5c-3e05-49ea-86e1-c218f4dd031f nodeName:}" failed. No retries permitted until 2026-07-29 10:38:45.798658334 +0000 UTC m=+638.787507768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls") pod "error-404-isvc-0a743-predictor-589d4b48df-5ss6c" (UID: "03c31e5c-3e05-49ea-86e1-c218f4dd031f") : secret "error-404-isvc-0a743-predictor-serving-cert" not found
Jul 29 10:38:45.299124 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.299103    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.309400 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.309369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwjkx\" (UniqueName: \"kubernetes.io/projected/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-api-access-vwjkx\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.490084 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.490049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:38:45.710509 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.710470    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerID="9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59" exitCode=2
Jul 29 10:38:45.710654 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.710525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" event={"ID":"dec3d21b-ccca-41e5-8be9-56c970cf4d5a","Type":"ContainerDied","Data":"9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59"}
Jul 29 10:38:45.805013 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.804902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:45.807397 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:45.807374    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls\") pod \"error-404-isvc-0a743-predictor-589d4b48df-5ss6c\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") " pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:46.010581 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.010536    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:46.148776 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.148750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"]
Jul 29 10:38:46.717686 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.717648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" event={"ID":"03c31e5c-3e05-49ea-86e1-c218f4dd031f","Type":"ContainerStarted","Data":"1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584"}
Jul 29 10:38:46.717896 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.717795    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:46.717896 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.717848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" event={"ID":"03c31e5c-3e05-49ea-86e1-c218f4dd031f","Type":"ContainerStarted","Data":"a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0"}
Jul 29 10:38:46.717896 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.717885    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" event={"ID":"03c31e5c-3e05-49ea-86e1-c218f4dd031f","Type":"ContainerStarted","Data":"e104fd7a964f55cdd200e8bdec01a98723a861103d9301c654c5b3073be33dd9"}
Jul 29 10:38:46.718033 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.717917    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:46.719501 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.719471    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:38:46.736594 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:46.736538    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podStartSLOduration=1.736521024 podStartE2EDuration="1.736521024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:38:46.734561551 +0000 UTC m=+639.723411008" watchObservedRunningTime="2026-07-29 10:38:46.736521024 +0000 UTC m=+639.725370481"
Jul 29 10:38:47.723674 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:47.723628    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:38:48.421056 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.421032    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:38:48.528797 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.528745    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6887\" (UniqueName: \"kubernetes.io/projected/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-api-access-s6887\") pod \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") "
Jul 29 10:38:48.528797 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.528808    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls\") pod \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") "
Jul 29 10:38:48.529026 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.528835    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-rbac-proxy-sar-config\") pod \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\" (UID: \"dec3d21b-ccca-41e5-8be9-56c970cf4d5a\") "
Jul 29 10:38:48.529257 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.529229    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "dec3d21b-ccca-41e5-8be9-56c970cf4d5a" (UID: "dec3d21b-ccca-41e5-8be9-56c970cf4d5a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:38:48.531122 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.531094    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dec3d21b-ccca-41e5-8be9-56c970cf4d5a" (UID: "dec3d21b-ccca-41e5-8be9-56c970cf4d5a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:38:48.531257 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.531139    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-api-access-s6887" (OuterVolumeSpecName: "kube-api-access-s6887") pod "dec3d21b-ccca-41e5-8be9-56c970cf4d5a" (UID: "dec3d21b-ccca-41e5-8be9-56c970cf4d5a"). InnerVolumeSpecName "kube-api-access-s6887". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:38:48.629429 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.629387    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:38:48.629429 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.629418    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6887\" (UniqueName: \"kubernetes.io/projected/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-kube-api-access-s6887\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:38:48.629429 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.629428    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dec3d21b-ccca-41e5-8be9-56c970cf4d5a-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:38:48.730227 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.730196    2562 generic.go:356] "Generic (PLEG): container finished" podID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerID="41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d" exitCode=0
Jul 29 10:38:48.730644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.730248    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"
Jul 29 10:38:48.730644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.730277    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" event={"ID":"dec3d21b-ccca-41e5-8be9-56c970cf4d5a","Type":"ContainerDied","Data":"41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d"}
Jul 29 10:38:48.730644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.730357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" event={"ID":"dec3d21b-ccca-41e5-8be9-56c970cf4d5a","Type":"ContainerDied","Data":"9b70bbee75491b230e026573668b10bfc88489565e3b06d6e4668297c0e8e601"}
Jul 29 10:38:48.730644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.730385    2562 scope.go:117] "RemoveContainer" containerID="9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59"
Jul 29 10:38:48.739832 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.739809    2562 scope.go:117] "RemoveContainer" containerID="41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d"
Jul 29 10:38:48.747723 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.747703    2562 scope.go:117] "RemoveContainer" containerID="9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59"
Jul 29 10:38:48.747974 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:38:48.747955    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59\": container with ID starting with 9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59 not found: ID does not exist" containerID="9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59"
Jul 29 10:38:48.748022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.747983    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59"} err="failed to get container status \"9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59\": rpc error: code = NotFound desc = could not find container \"9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59\": container with ID starting with 9ba96ea9395a5644796c12f78945f8fffc540e67002d5f49b5652069fa80ca59 not found: ID does not exist"
Jul 29 10:38:48.748022 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.748000    2562 scope.go:117] "RemoveContainer" containerID="41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d"
Jul 29 10:38:48.748234 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:38:48.748215    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d\": container with ID starting with 41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d not found: ID does not exist" containerID="41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d"
Jul 29 10:38:48.748273 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.748242    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d"} err="failed to get container status \"41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d\": rpc error: code = NotFound desc = could not find container \"41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d\": container with ID starting with 41cafc948f9de51335b7ddda0b0a265715d4276a4f050f314c1507e18533993d not found: ID does not exist"
Jul 29 10:38:48.754008 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.753982    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"]
Jul 29 10:38:48.757737 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:48.757716    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k"]
Jul 29 10:38:49.354254 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:49.354213    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 29 10:38:49.361088 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:49.361055    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0182-predictor-6ff7979689-6jg5k" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: i/o timeout"
Jul 29 10:38:49.506906 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:49.506875    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" path="/var/lib/kubelet/pods/dec3d21b-ccca-41e5-8be9-56c970cf4d5a/volumes"
Jul 29 10:38:52.728083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:52.728052    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:38:52.728516 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:38:52.728469    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:39:02.728999 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:02.728952    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:39:12.728638 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:12.728598    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:39:22.729056 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:22.729013    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:39:32.729520 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:32.729485    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:39:34.621377 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:34.621314    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:36.164604 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:36.164568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:44.725094 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.725045    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:44.822225 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.822190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:44.895515 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.895482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:44.990853 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.990815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:44.992042 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992019    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"]
Jul 29 10:39:44.992458 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992444    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kube-rbac-proxy"
Jul 29 10:39:44.992708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992459    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kube-rbac-proxy"
Jul 29 10:39:44.992708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992475    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container"
Jul 29 10:39:44.992708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992482    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container"
Jul 29 10:39:44.992708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992545    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kserve-container"
Jul 29 10:39:44.992708 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.992554    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dec3d21b-ccca-41e5-8be9-56c970cf4d5a" containerName="kube-rbac-proxy"
Jul 29 10:39:44.996004 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.995976    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:44.998563 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.998539    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f1d40-predictor-serving-cert\""
Jul 29 10:39:44.998694 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:44.998545    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f1d40-kube-rbac-proxy-sar-config\""
Jul 29 10:39:45.005442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.005410    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"]
Jul 29 10:39:45.057479 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.057430    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:45.116628 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.116595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:45.129547 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.129501    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-proxy-tls\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.129703 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.129590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.129703 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.129619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hgwt\" (UniqueName: \"kubernetes.io/projected/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-api-access-9hgwt\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.190750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.190709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:45.230975 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.230925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.231169 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.230984    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hgwt\" (UniqueName: \"kubernetes.io/projected/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-api-access-9hgwt\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.231169 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.231068    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-proxy-tls\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.231821 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.231796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.233567 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.233543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-proxy-tls\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.241890 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.241816    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hgwt\" (UniqueName: \"kubernetes.io/projected/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-api-access-9hgwt\") pod \"error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") " pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.291740 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.291706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:45.308150 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.308109    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:45.449676 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.449647    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"]
Jul 29 10:39:45.451883 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:39:45.451852    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7e36d2_0624_43cc_acbf_3c4117c8a4e7.slice/crio-2c5d2f5bf00bd92d5871404d7cc5af3c90439a704c334fcd6e5bbe8b2320a011 WatchSource:0}: Error finding container 2c5d2f5bf00bd92d5871404d7cc5af3c90439a704c334fcd6e5bbe8b2320a011: Status 404 returned error can't find the container with id 2c5d2f5bf00bd92d5871404d7cc5af3c90439a704c334fcd6e5bbe8b2320a011
Jul 29 10:39:45.795239 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:45.795147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:39:46.005455 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:46.005420    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" event={"ID":"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7","Type":"ContainerStarted","Data":"e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c"}
Jul 29 10:39:46.005670 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:46.005489    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" event={"ID":"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7","Type":"ContainerStarted","Data":"f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447"}
Jul 29 10:39:46.005670 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:46.005519    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" event={"ID":"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7","Type":"ContainerStarted","Data":"2c5d2f5bf00bd92d5871404d7cc5af3c90439a704c334fcd6e5bbe8b2320a011"}
Jul 29 10:39:46.005670 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:46.005550    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:46.023900 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:46.023833    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podStartSLOduration=2.023814245 podStartE2EDuration="2.023814245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:39:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:39:46.023070235 +0000 UTC m=+699.011919691" watchObservedRunningTime="2026-07-29 10:39:46.023814245 +0000 UTC m=+699.012663702"
Jul 29 10:39:47.010785 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:47.010756    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:47.012337 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:47.012296    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:39:48.015775 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:48.015732    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:39:53.020407 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:53.020373    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:39:53.020820 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:39:53.020796    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:40:03.021038 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:03.020944    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:40:13.021800 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:13.021758    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:40:16.477079 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:16.477046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:40:16.577934 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:16.577896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:40:23.021583 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:23.021541    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:40:33.022517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:33.022482    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:40:44.910059 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:44.910024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:40:45.022048 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:40:45.022011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:41:10.814616 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:41:10.814572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:41:33.059728 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:41:33.059630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:41:58.146010 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:41:58.145966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:43:07.526960 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:43:07.526929    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:43:07.530182 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:43:07.530156    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:46:10.001924 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:46:10.001828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:47:59.829442 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:47:59.829390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:47:59.927591 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:47:59.927554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:00.026583 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.026540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:00.027044 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.027023    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"]
Jul 29 10:48:00.030873 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.030851    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.033707 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.033678    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b12e8-predictor-serving-cert\""
Jul 29 10:48:00.033858 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.033686    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b12e8-kube-rbac-proxy-sar-config\""
Jul 29 10:48:00.042876 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.042845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"]
Jul 29 10:48:00.093820 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.093723    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"]
Jul 29 10:48:00.094013 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.093986    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" containerID="cri-o://a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0" gracePeriod=30
Jul 29 10:48:00.094089 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.094037    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kube-rbac-proxy" containerID="cri-o://1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584" gracePeriod=30
Jul 29 10:48:00.112976 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.112940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:00.190526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.190483    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71307829-fbab-456d-ba06-bccbc23e9076-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.190731 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.190541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6q98\" (UniqueName: \"kubernetes.io/projected/71307829-fbab-456d-ba06-bccbc23e9076-kube-api-access-h6q98\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.190731 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.190679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71307829-fbab-456d-ba06-bccbc23e9076-proxy-tls\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.212019 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.211980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:00.291879 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.291834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71307829-fbab-456d-ba06-bccbc23e9076-proxy-tls\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.291879 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.291879    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71307829-fbab-456d-ba06-bccbc23e9076-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.292127 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.291927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6q98\" (UniqueName: \"kubernetes.io/projected/71307829-fbab-456d-ba06-bccbc23e9076-kube-api-access-h6q98\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.292630 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.292607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71307829-fbab-456d-ba06-bccbc23e9076-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.294468 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.294442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71307829-fbab-456d-ba06-bccbc23e9076-proxy-tls\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.301075 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.301045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6q98\" (UniqueName: \"kubernetes.io/projected/71307829-fbab-456d-ba06-bccbc23e9076-kube-api-access-h6q98\") pod \"error-404-isvc-b12e8-predictor-665bb476f9-m2jk6\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") " pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.342397 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.342356    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:00.433681 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.433620    2562 generic.go:356] "Generic (PLEG): container finished" podID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerID="1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584" exitCode=2
Jul 29 10:48:00.433908 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.433671    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" event={"ID":"03c31e5c-3e05-49ea-86e1-c218f4dd031f","Type":"ContainerDied","Data":"1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584"}
Jul 29 10:48:00.486585 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:00.486555    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"]
Jul 29 10:48:00.488747 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:48:00.488713    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71307829_fbab_456d_ba06_bccbc23e9076.slice/crio-d98afe29af855dffba1d1ed02dd0b2753e6ada3b45cf25d1de5222c5f5ae188a WatchSource:0}: Error finding container d98afe29af855dffba1d1ed02dd0b2753e6ada3b45cf25d1de5222c5f5ae188a: Status 404 returned error can't find the container with id d98afe29af855dffba1d1ed02dd0b2753e6ada3b45cf25d1de5222c5f5ae188a
Jul 29 10:48:01.440427 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:01.440389    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" event={"ID":"71307829-fbab-456d-ba06-bccbc23e9076","Type":"ContainerStarted","Data":"5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf"}
Jul 29 10:48:01.440836 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:01.440448    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" event={"ID":"71307829-fbab-456d-ba06-bccbc23e9076","Type":"ContainerStarted","Data":"25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e"}
Jul 29 10:48:01.440836 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:01.440472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" event={"ID":"71307829-fbab-456d-ba06-bccbc23e9076","Type":"ContainerStarted","Data":"d98afe29af855dffba1d1ed02dd0b2753e6ada3b45cf25d1de5222c5f5ae188a"}
Jul 29 10:48:01.440836 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:01.440511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:01.459440 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:01.459372    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podStartSLOduration=1.459354517 podStartE2EDuration="1.459354517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:48:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:48:01.457557672 +0000 UTC m=+1194.446407120" watchObservedRunningTime="2026-07-29 10:48:01.459354517 +0000 UTC m=+1194.448203969"
Jul 29 10:48:02.445815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:02.445779    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:02.447498 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:02.447462    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 29 10:48:02.724182 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:02.724082    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 29 10:48:02.729542 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:02.729509    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 29 10:48:03.452187 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.452138    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 29 10:48:03.660487 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.660452    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:48:03.826747 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.826702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwjkx\" (UniqueName: \"kubernetes.io/projected/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-api-access-vwjkx\") pod \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") "
Jul 29 10:48:03.826961 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.826764    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls\") pod \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") "
Jul 29 10:48:03.826961 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.826831    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-rbac-proxy-sar-config\") pod \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\" (UID: \"03c31e5c-3e05-49ea-86e1-c218f4dd031f\") "
Jul 29 10:48:03.827211 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.827184    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "03c31e5c-3e05-49ea-86e1-c218f4dd031f" (UID: "03c31e5c-3e05-49ea-86e1-c218f4dd031f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:48:03.828973 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.828949    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "03c31e5c-3e05-49ea-86e1-c218f4dd031f" (UID: "03c31e5c-3e05-49ea-86e1-c218f4dd031f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:48:03.829141 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.829124    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-api-access-vwjkx" (OuterVolumeSpecName: "kube-api-access-vwjkx") pod "03c31e5c-3e05-49ea-86e1-c218f4dd031f" (UID: "03c31e5c-3e05-49ea-86e1-c218f4dd031f"). InnerVolumeSpecName "kube-api-access-vwjkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:48:03.928115 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.928071    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/03c31e5c-3e05-49ea-86e1-c218f4dd031f-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:48:03.928115 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.928106    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:48:03.928115 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:03.928116    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwjkx\" (UniqueName: \"kubernetes.io/projected/03c31e5c-3e05-49ea-86e1-c218f4dd031f-kube-api-access-vwjkx\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:48:04.457846 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.457811    2562 generic.go:356] "Generic (PLEG): container finished" podID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerID="a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0" exitCode=0
Jul 29 10:48:04.458266 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.457862    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"
Jul 29 10:48:04.458266 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.457879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" event={"ID":"03c31e5c-3e05-49ea-86e1-c218f4dd031f","Type":"ContainerDied","Data":"a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0"}
Jul 29 10:48:04.458266 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.457955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c" event={"ID":"03c31e5c-3e05-49ea-86e1-c218f4dd031f","Type":"ContainerDied","Data":"e104fd7a964f55cdd200e8bdec01a98723a861103d9301c654c5b3073be33dd9"}
Jul 29 10:48:04.458266 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.457980    2562 scope.go:117] "RemoveContainer" containerID="1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584"
Jul 29 10:48:04.467744 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.467722    2562 scope.go:117] "RemoveContainer" containerID="a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0"
Jul 29 10:48:04.475704 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.475683    2562 scope.go:117] "RemoveContainer" containerID="1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584"
Jul 29 10:48:04.475996 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:48:04.475977    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584\": container with ID starting with 1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584 not found: ID does not exist" containerID="1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584"
Jul 29 10:48:04.476064 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.476010    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584"} err="failed to get container status \"1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584\": rpc error: code = NotFound desc = could not find container \"1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584\": container with ID starting with 1d6e0fb9293a17f39d85e6845e4f0a103d24a5b2a34286481c09e67d27c65584 not found: ID does not exist"
Jul 29 10:48:04.476064 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.476037    2562 scope.go:117] "RemoveContainer" containerID="a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0"
Jul 29 10:48:04.476294 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:48:04.476276    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0\": container with ID starting with a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0 not found: ID does not exist" containerID="a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0"
Jul 29 10:48:04.476361 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.476302    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0"} err="failed to get container status \"a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0\": rpc error: code = NotFound desc = could not find container \"a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0\": container with ID starting with a21336dad3da384c9fd97cd8ca28a7a97011d4900a709ee68c5e432ad41a0aa0 not found: ID does not exist"
Jul 29 10:48:04.481795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.481766    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"]
Jul 29 10:48:04.483805 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:04.483781    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0a743-predictor-589d4b48df-5ss6c"]
Jul 29 10:48:05.507645 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:05.507613    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" path="/var/lib/kubelet/pods/03c31e5c-3e05-49ea-86e1-c218f4dd031f/volumes"
Jul 29 10:48:07.561107 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:07.561079    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:48:07.565182 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:07.565160    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:48:08.456579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:08.456549    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:08.456991 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:08.456958    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 29 10:48:18.457111 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:18.457071    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 29 10:48:28.457370 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:28.457306    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 29 10:48:38.458047 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:38.458007    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 29 10:48:48.457904 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:48.457871    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:48:59.614386 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.614285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:59.677490 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.677454    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"]
Jul 29 10:48:59.677716 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.677688    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" containerID="cri-o://f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447" gracePeriod=30
Jul 29 10:48:59.677865 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.677832    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kube-rbac-proxy" containerID="cri-o://e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c" gracePeriod=30
Jul 29 10:48:59.712518 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.712478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:59.787550 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.787519    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:59.867501 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.867385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:48:59.881092 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881054    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"]
Jul 29 10:48:59.881614 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881592    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container"
Jul 29 10:48:59.881614 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881615    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container"
Jul 29 10:48:59.881795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881651    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kube-rbac-proxy"
Jul 29 10:48:59.881795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881659    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kube-rbac-proxy"
Jul 29 10:48:59.881795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881759    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kube-rbac-proxy"
Jul 29 10:48:59.881795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.881776    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="03c31e5c-3e05-49ea-86e1-c218f4dd031f" containerName="kserve-container"
Jul 29 10:48:59.885060 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.885032    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:48:59.887760 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.887732    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6242a-kube-rbac-proxy-sar-config\""
Jul 29 10:48:59.887991 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.887962    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6242a-predictor-serving-cert\""
Jul 29 10:48:59.894630 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.894598    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"]
Jul 29 10:48:59.938177 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.938140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:48:59.938367 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.938258    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8676s\" (UniqueName: \"kubernetes.io/projected/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-api-access-8676s\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:48:59.938367 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.938299    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/599d1b04-e12c-469f-abd7-05967a2b1f8f-proxy-tls\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:48:59.969420 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:48:59.969376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:00.038876 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.038829    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8676s\" (UniqueName: \"kubernetes.io/projected/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-api-access-8676s\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.039084 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.038891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/599d1b04-e12c-469f-abd7-05967a2b1f8f-proxy-tls\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.039084 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.038942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.039689 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.039662    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.041740 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.041714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/599d1b04-e12c-469f-abd7-05967a2b1f8f-proxy-tls\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.047411 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.047381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:00.050915 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.050884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8676s\" (UniqueName: \"kubernetes.io/projected/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-api-access-8676s\") pod \"error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") " pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.149358 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.149246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:00.198716 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.198655    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.342148 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.342123    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"]
Jul 29 10:49:00.344314 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:49:00.344287    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod599d1b04_e12c_469f_abd7_05967a2b1f8f.slice/crio-4640005a4144225db6ea0060c646e4f5bb9fb6b8e9135461f02e16defe5df0bc WatchSource:0}: Error finding container 4640005a4144225db6ea0060c646e4f5bb9fb6b8e9135461f02e16defe5df0bc: Status 404 returned error can't find the container with id 4640005a4144225db6ea0060c646e4f5bb9fb6b8e9135461f02e16defe5df0bc
Jul 29 10:49:00.569221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.569181    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:00.732574 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.732539    2562 generic.go:356] "Generic (PLEG): container finished" podID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerID="e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c" exitCode=2
Jul 29 10:49:00.733003 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.732611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" event={"ID":"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7","Type":"ContainerDied","Data":"e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c"}
Jul 29 10:49:00.734440 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.734417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" event={"ID":"599d1b04-e12c-469f-abd7-05967a2b1f8f","Type":"ContainerStarted","Data":"60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197"}
Jul 29 10:49:00.734558 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.734463    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" event={"ID":"599d1b04-e12c-469f-abd7-05967a2b1f8f","Type":"ContainerStarted","Data":"3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372"}
Jul 29 10:49:00.734558 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.734484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" event={"ID":"599d1b04-e12c-469f-abd7-05967a2b1f8f","Type":"ContainerStarted","Data":"4640005a4144225db6ea0060c646e4f5bb9fb6b8e9135461f02e16defe5df0bc"}
Jul 29 10:49:00.734678 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.734610    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:00.756769 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:00.756717    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podStartSLOduration=1.756699367 podStartE2EDuration="1.756699367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:49:00.754434332 +0000 UTC m=+1253.743283788" watchObservedRunningTime="2026-07-29 10:49:00.756699367 +0000 UTC m=+1253.745548822"
Jul 29 10:49:01.738992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:01.738952    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:01.740359 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:01.740316    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 29 10:49:02.744023 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:02.743979    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 29 10:49:03.016267 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.016153    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 29 10:49:03.021716 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.021663    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 29 10:49:03.137008 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.136975    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:49:03.160892 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.160859    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-proxy-tls\") pod \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") "
Jul 29 10:49:03.161089 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.160906    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hgwt\" (UniqueName: \"kubernetes.io/projected/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-api-access-9hgwt\") pod \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") "
Jul 29 10:49:03.161089 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.160949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-rbac-proxy-sar-config\") pod \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\" (UID: \"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7\") "
Jul 29 10:49:03.161428 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.161397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" (UID: "9c7e36d2-0624-43cc-acbf-3c4117c8a4e7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:49:03.163034 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.163002    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" (UID: "9c7e36d2-0624-43cc-acbf-3c4117c8a4e7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:49:03.163199 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.163174    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-api-access-9hgwt" (OuterVolumeSpecName: "kube-api-access-9hgwt") pod "9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" (UID: "9c7e36d2-0624-43cc-acbf-3c4117c8a4e7"). InnerVolumeSpecName "kube-api-access-9hgwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:49:03.261745 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.261696    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:49:03.261745 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.261736    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hgwt\" (UniqueName: \"kubernetes.io/projected/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-api-access-9hgwt\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:49:03.261745 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.261750    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:49:03.749798 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.749762    2562 generic.go:356] "Generic (PLEG): container finished" podID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerID="f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447" exitCode=0
Jul 29 10:49:03.750233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.749809    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"
Jul 29 10:49:03.750233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.749842    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" event={"ID":"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7","Type":"ContainerDied","Data":"f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447"}
Jul 29 10:49:03.750233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.749909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj" event={"ID":"9c7e36d2-0624-43cc-acbf-3c4117c8a4e7","Type":"ContainerDied","Data":"2c5d2f5bf00bd92d5871404d7cc5af3c90439a704c334fcd6e5bbe8b2320a011"}
Jul 29 10:49:03.750233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.749935    2562 scope.go:117] "RemoveContainer" containerID="e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c"
Jul 29 10:49:03.758374 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.758352    2562 scope.go:117] "RemoveContainer" containerID="f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447"
Jul 29 10:49:03.766526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.766506    2562 scope.go:117] "RemoveContainer" containerID="e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c"
Jul 29 10:49:03.766865 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:49:03.766836    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c\": container with ID starting with e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c not found: ID does not exist" containerID="e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c"
Jul 29 10:49:03.766974 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.766870    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c"} err="failed to get container status \"e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c\": rpc error: code = NotFound desc = could not find container \"e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c\": container with ID starting with e3a6a52a0d83650d3ab4fffa09bc52783bd37f226c5fa09c3c66d297059a593c not found: ID does not exist"
Jul 29 10:49:03.766974 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.766891    2562 scope.go:117] "RemoveContainer" containerID="f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447"
Jul 29 10:49:03.767226 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:49:03.767201    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447\": container with ID starting with f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447 not found: ID does not exist" containerID="f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447"
Jul 29 10:49:03.767314 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.767231    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447"} err="failed to get container status \"f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447\": rpc error: code = NotFound desc = could not find container \"f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447\": container with ID starting with f07708783f2b1aabb5b1357730a78061a60fcf03b4d428968b325c4b063d6447 not found: ID does not exist"
Jul 29 10:49:03.769662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.769642    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"]
Jul 29 10:49:03.773526 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:03.773506    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f1d40-predictor-749bd5f9df-wqmpj"]
Jul 29 10:49:05.507042 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:05.507006    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" path="/var/lib/kubelet/pods/9c7e36d2-0624-43cc-acbf-3c4117c8a4e7/volumes"
Jul 29 10:49:07.749020 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:07.748993    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:07.749502 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:07.749470    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 29 10:49:10.127236 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.127141    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:10.185939 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.185903    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"]
Jul 29 10:49:10.186220 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.186191    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container" containerID="cri-o://25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e" gracePeriod=30
Jul 29 10:49:10.186337 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.186224    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kube-rbac-proxy" containerID="cri-o://5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf" gracePeriod=30
Jul 29 10:49:10.212994 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.212955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:10.289132 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.289080    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:10.357690 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.357638    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:10.358853 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.358824    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"]
Jul 29 10:49:10.359355 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.359315    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kube-rbac-proxy"
Jul 29 10:49:10.359435 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.359358    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kube-rbac-proxy"
Jul 29 10:49:10.359435 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.359372    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container"
Jul 29 10:49:10.359435 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.359377    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container"
Jul 29 10:49:10.359534 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.359471    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kube-rbac-proxy"
Jul 29 10:49:10.359534 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.359481    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c7e36d2-0624-43cc-acbf-3c4117c8a4e7" containerName="kserve-container"
Jul 29 10:49:10.362664 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.362646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.365662 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.365640    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d2d23-kube-rbac-proxy-sar-config\""
Jul 29 10:49:10.365757 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.365639    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d2d23-predictor-serving-cert\""
Jul 29 10:49:10.373012 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.372985    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"]
Jul 29 10:49:10.424082 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.424000    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qrx2\" (UniqueName: \"kubernetes.io/projected/53028cc2-acf8-4064-81d6-76c1a003da51-kube-api-access-4qrx2\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.424082 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.424067    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/53028cc2-acf8-4064-81d6-76c1a003da51-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.424274 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.424105    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/53028cc2-acf8-4064-81d6-76c1a003da51-proxy-tls\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.521916 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.521876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:10.524931 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.524908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qrx2\" (UniqueName: \"kubernetes.io/projected/53028cc2-acf8-4064-81d6-76c1a003da51-kube-api-access-4qrx2\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.525013 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.524959    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/53028cc2-acf8-4064-81d6-76c1a003da51-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.525013 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.524980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/53028cc2-acf8-4064-81d6-76c1a003da51-proxy-tls\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.525685 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.525668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/53028cc2-acf8-4064-81d6-76c1a003da51-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.527388 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.527369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/53028cc2-acf8-4064-81d6-76c1a003da51-proxy-tls\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.533307 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.533280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qrx2\" (UniqueName: \"kubernetes.io/projected/53028cc2-acf8-4064-81d6-76c1a003da51-kube-api-access-4qrx2\") pod \"error-404-isvc-d2d23-predictor-658b5fdc7-kx72c\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") " pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.674844 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.674737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:10.762645 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.762610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:10.790809 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.790775    2562 generic.go:356] "Generic (PLEG): container finished" podID="71307829-fbab-456d-ba06-bccbc23e9076" containerID="5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf" exitCode=2
Jul 29 10:49:10.790989 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.790848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" event={"ID":"71307829-fbab-456d-ba06-bccbc23e9076","Type":"ContainerDied","Data":"5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf"}
Jul 29 10:49:10.817437 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:10.817379    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"]
Jul 29 10:49:10.819734 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:49:10.819703    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53028cc2_acf8_4064_81d6_76c1a003da51.slice/crio-d3b9c0defa069574b7fddcaefa635d180ed9490461c3c677ce66095649362f7c WatchSource:0}: Error finding container d3b9c0defa069574b7fddcaefa635d180ed9490461c3c677ce66095649362f7c: Status 404 returned error can't find the container with id d3b9c0defa069574b7fddcaefa635d180ed9490461c3c677ce66095649362f7c
Jul 29 10:49:11.799126 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:11.799083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" event={"ID":"53028cc2-acf8-4064-81d6-76c1a003da51","Type":"ContainerStarted","Data":"7c9e5706c588c38415c6cf9ffcca807ea513a681a9e2400648e7287c524db1b6"}
Jul 29 10:49:11.799673 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:11.799168    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" event={"ID":"53028cc2-acf8-4064-81d6-76c1a003da51","Type":"ContainerStarted","Data":"5c17a964ee9e451d69a247681ccdf0e8f62c5c46a52546391610ed6337007a0c"}
Jul 29 10:49:11.799673 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:11.799200    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" event={"ID":"53028cc2-acf8-4064-81d6-76c1a003da51","Type":"ContainerStarted","Data":"d3b9c0defa069574b7fddcaefa635d180ed9490461c3c677ce66095649362f7c"}
Jul 29 10:49:11.799673 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:11.799235    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:11.818626 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:11.818568    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podStartSLOduration=1.8185500239999999 podStartE2EDuration="1.818550024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:49:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:49:11.816471104 +0000 UTC m=+1264.805320561" watchObservedRunningTime="2026-07-29 10:49:11.818550024 +0000 UTC m=+1264.807399478"
Jul 29 10:49:12.811939 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:12.811903    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:12.813587 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:12.813554    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 29 10:49:13.452818 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.452767    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Jul 29 10:49:13.741771 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.741742    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:49:13.817957 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.817917    2562 generic.go:356] "Generic (PLEG): container finished" podID="71307829-fbab-456d-ba06-bccbc23e9076" containerID="25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e" exitCode=0
Jul 29 10:49:13.818418 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.817975    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"
Jul 29 10:49:13.818418 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.818000    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" event={"ID":"71307829-fbab-456d-ba06-bccbc23e9076","Type":"ContainerDied","Data":"25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e"}
Jul 29 10:49:13.818418 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.818073    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6" event={"ID":"71307829-fbab-456d-ba06-bccbc23e9076","Type":"ContainerDied","Data":"d98afe29af855dffba1d1ed02dd0b2753e6ada3b45cf25d1de5222c5f5ae188a"}
Jul 29 10:49:13.818418 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.818104    2562 scope.go:117] "RemoveContainer" containerID="5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf"
Jul 29 10:49:13.818622 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.818581    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 29 10:49:13.826732 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.826710    2562 scope.go:117] "RemoveContainer" containerID="25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e"
Jul 29 10:49:13.834682 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.834661    2562 scope.go:117] "RemoveContainer" containerID="5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf"
Jul 29 10:49:13.835024 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:49:13.835001    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf\": container with ID starting with 5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf not found: ID does not exist" containerID="5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf"
Jul 29 10:49:13.835127 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.835029    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf"} err="failed to get container status \"5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf\": rpc error: code = NotFound desc = could not find container \"5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf\": container with ID starting with 5ec030c134946e7b03e8ad65a609afc487cfacf4ee457398ca301f56e69b36bf not found: ID does not exist"
Jul 29 10:49:13.835127 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.835050    2562 scope.go:117] "RemoveContainer" containerID="25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e"
Jul 29 10:49:13.835307 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:49:13.835289    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e\": container with ID starting with 25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e not found: ID does not exist" containerID="25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e"
Jul 29 10:49:13.835469 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.835313    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e"} err="failed to get container status \"25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e\": rpc error: code = NotFound desc = could not find container \"25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e\": container with ID starting with 25e504adfab823e8dc2ec70ed7d8ec710db1f7d48d7ae54e360f018aaaf1f13e not found: ID does not exist"
Jul 29 10:49:13.855140 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.855102    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71307829-fbab-456d-ba06-bccbc23e9076-proxy-tls\") pod \"71307829-fbab-456d-ba06-bccbc23e9076\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") "
Jul 29 10:49:13.855140 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.855139    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6q98\" (UniqueName: \"kubernetes.io/projected/71307829-fbab-456d-ba06-bccbc23e9076-kube-api-access-h6q98\") pod \"71307829-fbab-456d-ba06-bccbc23e9076\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") "
Jul 29 10:49:13.855408 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.855177    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71307829-fbab-456d-ba06-bccbc23e9076-kube-rbac-proxy-sar-config\") pod \"71307829-fbab-456d-ba06-bccbc23e9076\" (UID: \"71307829-fbab-456d-ba06-bccbc23e9076\") "
Jul 29 10:49:13.855763 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.855731    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71307829-fbab-456d-ba06-bccbc23e9076-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "71307829-fbab-456d-ba06-bccbc23e9076" (UID: "71307829-fbab-456d-ba06-bccbc23e9076"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:49:13.857221 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.857199    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71307829-fbab-456d-ba06-bccbc23e9076-kube-api-access-h6q98" (OuterVolumeSpecName: "kube-api-access-h6q98") pod "71307829-fbab-456d-ba06-bccbc23e9076" (UID: "71307829-fbab-456d-ba06-bccbc23e9076"). InnerVolumeSpecName "kube-api-access-h6q98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:49:13.857459 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.857442    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71307829-fbab-456d-ba06-bccbc23e9076-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "71307829-fbab-456d-ba06-bccbc23e9076" (UID: "71307829-fbab-456d-ba06-bccbc23e9076"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:49:13.956669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.956628    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/71307829-fbab-456d-ba06-bccbc23e9076-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:49:13.956669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.956666    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6q98\" (UniqueName: \"kubernetes.io/projected/71307829-fbab-456d-ba06-bccbc23e9076-kube-api-access-h6q98\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:49:13.956669 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:13.956679    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/71307829-fbab-456d-ba06-bccbc23e9076-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:49:14.142468 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:14.142432    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"]
Jul 29 10:49:14.147967 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:14.147936    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b12e8-predictor-665bb476f9-m2jk6"]
Jul 29 10:49:15.507684 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:15.507651    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71307829-fbab-456d-ba06-bccbc23e9076" path="/var/lib/kubelet/pods/71307829-fbab-456d-ba06-bccbc23e9076/volumes"
Jul 29 10:49:17.750449 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:17.750409    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 29 10:49:18.823420 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:18.823385    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:18.824003 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:18.823978    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 29 10:49:27.750309 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:27.750262    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 29 10:49:28.824781 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:28.824738    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 29 10:49:32.429154 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:32.429122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:49:37.749578 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:37.749535    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 29 10:49:38.824223 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:38.824184    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 29 10:49:47.750513 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:47.750481    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:49:48.824666 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:48.824625    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 29 10:49:58.825511 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:58.825476    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:49:59.825233 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:49:59.825184    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:00.331402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:00.331366    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:09.911042 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:09.911003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:10.056053 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.055967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:10.063841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.063814    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"]
Jul 29 10:50:10.064105 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.064076    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container" containerID="cri-o://3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372" gracePeriod=30
Jul 29 10:50:10.064252 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.064147    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kube-rbac-proxy" containerID="cri-o://60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197" gracePeriod=30
Jul 29 10:50:10.140689 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.140651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:10.146795 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.146763    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"]
Jul 29 10:50:10.147268 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.147249    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kube-rbac-proxy"
Jul 29 10:50:10.147402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.147269    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kube-rbac-proxy"
Jul 29 10:50:10.147402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.147285    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container"
Jul 29 10:50:10.147402 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.147293    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container"
Jul 29 10:50:10.147580 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.147473    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kserve-container"
Jul 29 10:50:10.147580 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.147489    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="71307829-fbab-456d-ba06-bccbc23e9076" containerName="kube-rbac-proxy"
Jul 29 10:50:10.151812 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.151792    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.154373 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.154348    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0934b-predictor-serving-cert\""
Jul 29 10:50:10.154543 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.154462    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0934b-kube-rbac-proxy-sar-config\""
Jul 29 10:50:10.161711 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.161689    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"]
Jul 29 10:50:10.235667 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.235631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:10.267161 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.267122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.267384 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.267291    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-proxy-tls\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.267384 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.267348    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6d5k\" (UniqueName: \"kubernetes.io/projected/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-api-access-c6d5k\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.313117 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.313025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:10.368311 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.368276    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-proxy-tls\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.368311 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.368348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6d5k\" (UniqueName: \"kubernetes.io/projected/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-api-access-c6d5k\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.368579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.368429    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.369112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.369094    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.370873 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.370852    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-proxy-tls\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.381901 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.381873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6d5k\" (UniqueName: \"kubernetes.io/projected/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-api-access-c6d5k\") pod \"error-404-isvc-0934b-predictor-5949bbfc75-kftfs\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") " pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.425106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.425074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:10.462900 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.462863    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:10.606651 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.606623    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"]
Jul 29 10:50:10.609173 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:50:10.609139    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8afd5e_7b5f_4b67_b55b_fb8f8dfccbfc.slice/crio-eb69c3d99192e96402d7887ad3ddf1cae518f33e79a75955c0c43814c4d0074e WatchSource:0}: Error finding container eb69c3d99192e96402d7887ad3ddf1cae518f33e79a75955c0c43814c4d0074e: Status 404 returned error can't find the container with id eb69c3d99192e96402d7887ad3ddf1cae518f33e79a75955c0c43814c4d0074e
Jul 29 10:50:10.977182 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:10.977151    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:50:11.103678 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.103639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" event={"ID":"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc","Type":"ContainerStarted","Data":"f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b"}
Jul 29 10:50:11.103879 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.103705    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" event={"ID":"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc","Type":"ContainerStarted","Data":"4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4"}
Jul 29 10:50:11.103879 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.103736    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" event={"ID":"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc","Type":"ContainerStarted","Data":"eb69c3d99192e96402d7887ad3ddf1cae518f33e79a75955c0c43814c4d0074e"}
Jul 29 10:50:11.103879 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.103767    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:11.103879 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.103796    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:11.105624 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.105591    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:50:11.105969 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.105912    2562 generic.go:356] "Generic (PLEG): container finished" podID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerID="60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197" exitCode=2
Jul 29 10:50:11.106070 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.105992    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" event={"ID":"599d1b04-e12c-469f-abd7-05967a2b1f8f","Type":"ContainerDied","Data":"60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197"}
Jul 29 10:50:11.123140 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:11.123072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podStartSLOduration=1.1230572620000001 podStartE2EDuration="1.123057262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:50:11.1211492 +0000 UTC m=+1324.109998656" watchObservedRunningTime="2026-07-29 10:50:11.123057262 +0000 UTC m=+1324.111906717"
Jul 29 10:50:12.112112 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:12.112066    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:50:12.744777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:12.744728    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Jul 29 10:50:13.427583 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.427557    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:50:13.494131 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.494095    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/599d1b04-e12c-469f-abd7-05967a2b1f8f-proxy-tls\") pod \"599d1b04-e12c-469f-abd7-05967a2b1f8f\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") "
Jul 29 10:50:13.494365 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.494167    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-rbac-proxy-sar-config\") pod \"599d1b04-e12c-469f-abd7-05967a2b1f8f\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") "
Jul 29 10:50:13.494365 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.494308    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8676s\" (UniqueName: \"kubernetes.io/projected/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-api-access-8676s\") pod \"599d1b04-e12c-469f-abd7-05967a2b1f8f\" (UID: \"599d1b04-e12c-469f-abd7-05967a2b1f8f\") "
Jul 29 10:50:13.494566 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.494542    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "599d1b04-e12c-469f-abd7-05967a2b1f8f" (UID: "599d1b04-e12c-469f-abd7-05967a2b1f8f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:50:13.494654 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.494640    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:50:13.496248 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.496223    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-api-access-8676s" (OuterVolumeSpecName: "kube-api-access-8676s") pod "599d1b04-e12c-469f-abd7-05967a2b1f8f" (UID: "599d1b04-e12c-469f-abd7-05967a2b1f8f"). InnerVolumeSpecName "kube-api-access-8676s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:50:13.496459 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.496283    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/599d1b04-e12c-469f-abd7-05967a2b1f8f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "599d1b04-e12c-469f-abd7-05967a2b1f8f" (UID: "599d1b04-e12c-469f-abd7-05967a2b1f8f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:50:13.595751 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.595712    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/599d1b04-e12c-469f-abd7-05967a2b1f8f-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:50:13.595751 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:13.595744    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8676s\" (UniqueName: \"kubernetes.io/projected/599d1b04-e12c-469f-abd7-05967a2b1f8f-kube-api-access-8676s\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:50:14.125809 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.125776    2562 generic.go:356] "Generic (PLEG): container finished" podID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerID="3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372" exitCode=0
Jul 29 10:50:14.125992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.125822    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"
Jul 29 10:50:14.125992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.125839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" event={"ID":"599d1b04-e12c-469f-abd7-05967a2b1f8f","Type":"ContainerDied","Data":"3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372"}
Jul 29 10:50:14.125992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.125891    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr" event={"ID":"599d1b04-e12c-469f-abd7-05967a2b1f8f","Type":"ContainerDied","Data":"4640005a4144225db6ea0060c646e4f5bb9fb6b8e9135461f02e16defe5df0bc"}
Jul 29 10:50:14.125992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.125922    2562 scope.go:117] "RemoveContainer" containerID="60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197"
Jul 29 10:50:14.134634 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.134612    2562 scope.go:117] "RemoveContainer" containerID="3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372"
Jul 29 10:50:14.143008 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.142966    2562 scope.go:117] "RemoveContainer" containerID="60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197"
Jul 29 10:50:14.143399 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:50:14.143346    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197\": container with ID starting with 60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197 not found: ID does not exist" containerID="60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197"
Jul 29 10:50:14.143399 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.143387    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197"} err="failed to get container status \"60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197\": rpc error: code = NotFound desc = could not find container \"60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197\": container with ID starting with 60ce720b85772971b3102c2583b5f2426f56f547cfa0d1315b7f32c5c15e7197 not found: ID does not exist"
Jul 29 10:50:14.143562 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.143417    2562 scope.go:117] "RemoveContainer" containerID="3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372"
Jul 29 10:50:14.143707 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:50:14.143688    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372\": container with ID starting with 3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372 not found: ID does not exist" containerID="3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372"
Jul 29 10:50:14.143784 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.143712    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372"} err="failed to get container status \"3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372\": rpc error: code = NotFound desc = could not find container \"3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372\": container with ID starting with 3d778414751b3e0f43b7038fa772c8067adfb2827921b4ad9eda60845e14e372 not found: ID does not exist"
Jul 29 10:50:14.145645 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.145623    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"]
Jul 29 10:50:14.147368 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:14.147348    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6242a-predictor-6d94cbd9cc-4qrtr"]
Jul 29 10:50:15.507009 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:15.506972    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" path="/var/lib/kubelet/pods/599d1b04-e12c-469f-abd7-05967a2b1f8f/volumes"
Jul 29 10:50:17.116195 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:17.116160    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:50:17.116834 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:17.116799    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:50:27.117537 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:27.117495    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:50:37.117336 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:37.117236    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:50:47.117522 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:47.117470    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:50:57.117291 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:50:57.117258    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:51:10.142216 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:51:10.142132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:51:10.239153 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:51:10.239111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:53:07.591403 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:53:07.591373    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:53:07.597177 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:53:07.597144    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:55:13.078791 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:55:13.078711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:55:27.325445 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:55:27.325408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:07.625535 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:07.625494    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:58:07.631002 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:07.630981    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 10:58:24.907644 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:24.907588    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.055602 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.055571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.143913 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.143883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.197680 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.197593    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"]
Jul 29 10:58:25.198057 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.198040    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kube-rbac-proxy"
Jul 29 10:58:25.198106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.198060    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kube-rbac-proxy"
Jul 29 10:58:25.198106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.198084    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container"
Jul 29 10:58:25.198106 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.198090    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container"
Jul 29 10:58:25.198211 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.198146    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kube-rbac-proxy"
Jul 29 10:58:25.198211 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.198160    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="599d1b04-e12c-469f-abd7-05967a2b1f8f" containerName="kserve-container"
Jul 29 10:58:25.201408 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.201388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.204114 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.204091    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f5066-kube-rbac-proxy-sar-config\""
Jul 29 10:58:25.204235 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.204182    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f5066-predictor-serving-cert\""
Jul 29 10:58:25.211288 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.211251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.212453 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.212430    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"]
Jul 29 10:58:25.233992 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.233962    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"]
Jul 29 10:58:25.234356 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.234280    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container" containerID="cri-o://5c17a964ee9e451d69a247681ccdf0e8f62c5c46a52546391610ed6337007a0c" gracePeriod=30
Jul 29 10:58:25.234549 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.234433    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kube-rbac-proxy" containerID="cri-o://7c9e5706c588c38415c6cf9ffcca807ea513a681a9e2400648e7287c524db1b6" gracePeriod=30
Jul 29 10:58:25.263016 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.262982    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.305083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.301591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-proxy-tls\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.305083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.301802    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.305083 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.301846    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xndzz\" (UniqueName: \"kubernetes.io/projected/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-api-access-xndzz\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.333334 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.333293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.402741 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.402703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-proxy-tls\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.402944 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.402779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.402944 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.402803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xndzz\" (UniqueName: \"kubernetes.io/projected/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-api-access-xndzz\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.403574 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.403547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.405276 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.405258    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-proxy-tls\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.411341 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.411290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xndzz\" (UniqueName: \"kubernetes.io/projected/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-api-access-xndzz\") pod \"error-404-isvc-f5066-predictor-645f7d7b85-fsdjq\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") " pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.434919 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.434886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:58:25.501380 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.501343    2562 generic.go:356] "Generic (PLEG): container finished" podID="53028cc2-acf8-4064-81d6-76c1a003da51" containerID="7c9e5706c588c38415c6cf9ffcca807ea513a681a9e2400648e7287c524db1b6" exitCode=2
Jul 29 10:58:25.507733 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.507688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" event={"ID":"53028cc2-acf8-4064-81d6-76c1a003da51","Type":"ContainerDied","Data":"7c9e5706c588c38415c6cf9ffcca807ea513a681a9e2400648e7287c524db1b6"}
Jul 29 10:58:25.514486 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.514453    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:25.664573 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:25.664546    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"]
Jul 29 10:58:25.666611 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:58:25.666585    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b77336e_7fa8_40fa_8886_cbf3b2b9b729.slice/crio-f8b2f7f753c012febb430b3fe15ee2a95ddb34b500895efa1e8a394d6e3ef8d1 WatchSource:0}: Error finding container f8b2f7f753c012febb430b3fe15ee2a95ddb34b500895efa1e8a394d6e3ef8d1: Status 404 returned error can't find the container with id f8b2f7f753c012febb430b3fe15ee2a95ddb34b500895efa1e8a394d6e3ef8d1
Jul 29 10:58:26.511391 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:26.511347    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" event={"ID":"5b77336e-7fa8-40fa-8886-cbf3b2b9b729","Type":"ContainerStarted","Data":"feb8a433e344b7ad62a8422471259964f52427c02a0a9062967bb7311fb146b9"}
Jul 29 10:58:26.511807 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:26.511417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" event={"ID":"5b77336e-7fa8-40fa-8886-cbf3b2b9b729","Type":"ContainerStarted","Data":"32aae38b912eac31428200207be3ed51f08fc33666a6503c8f880d6b92265a21"}
Jul 29 10:58:26.511807 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:26.511456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:26.511807 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:26.511481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" event={"ID":"5b77336e-7fa8-40fa-8886-cbf3b2b9b729","Type":"ContainerStarted","Data":"f8b2f7f753c012febb430b3fe15ee2a95ddb34b500895efa1e8a394d6e3ef8d1"}
Jul 29 10:58:26.530573 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:26.530521    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podStartSLOduration=1.5305021079999999 podStartE2EDuration="1.530502108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:58:26.528435081 +0000 UTC m=+1819.517284547" watchObservedRunningTime="2026-07-29 10:58:26.530502108 +0000 UTC m=+1819.519351564"
Jul 29 10:58:27.516395 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:27.516362    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:27.518005 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:27.517974    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 29 10:58:28.526761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.526727    2562 generic.go:356] "Generic (PLEG): container finished" podID="53028cc2-acf8-4064-81d6-76c1a003da51" containerID="5c17a964ee9e451d69a247681ccdf0e8f62c5c46a52546391610ed6337007a0c" exitCode=0
Jul 29 10:58:28.527180 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.526812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" event={"ID":"53028cc2-acf8-4064-81d6-76c1a003da51","Type":"ContainerDied","Data":"5c17a964ee9e451d69a247681ccdf0e8f62c5c46a52546391610ed6337007a0c"}
Jul 29 10:58:28.527180 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.527114    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 29 10:58:28.578147 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.578122    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:58:28.734153 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.734122    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/53028cc2-acf8-4064-81d6-76c1a003da51-proxy-tls\") pod \"53028cc2-acf8-4064-81d6-76c1a003da51\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") "
Jul 29 10:58:28.734308 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.734195    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qrx2\" (UniqueName: \"kubernetes.io/projected/53028cc2-acf8-4064-81d6-76c1a003da51-kube-api-access-4qrx2\") pod \"53028cc2-acf8-4064-81d6-76c1a003da51\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") "
Jul 29 10:58:28.734308 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.734223    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/53028cc2-acf8-4064-81d6-76c1a003da51-kube-rbac-proxy-sar-config\") pod \"53028cc2-acf8-4064-81d6-76c1a003da51\" (UID: \"53028cc2-acf8-4064-81d6-76c1a003da51\") "
Jul 29 10:58:28.734672 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.734645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53028cc2-acf8-4064-81d6-76c1a003da51-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "53028cc2-acf8-4064-81d6-76c1a003da51" (UID: "53028cc2-acf8-4064-81d6-76c1a003da51"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:58:28.736367 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.736345    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53028cc2-acf8-4064-81d6-76c1a003da51-kube-api-access-4qrx2" (OuterVolumeSpecName: "kube-api-access-4qrx2") pod "53028cc2-acf8-4064-81d6-76c1a003da51" (UID: "53028cc2-acf8-4064-81d6-76c1a003da51"). InnerVolumeSpecName "kube-api-access-4qrx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:58:28.736476 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.736363    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53028cc2-acf8-4064-81d6-76c1a003da51-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "53028cc2-acf8-4064-81d6-76c1a003da51" (UID: "53028cc2-acf8-4064-81d6-76c1a003da51"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:58:28.835208 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.835149    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/53028cc2-acf8-4064-81d6-76c1a003da51-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:58:28.835208 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.835210    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qrx2\" (UniqueName: \"kubernetes.io/projected/53028cc2-acf8-4064-81d6-76c1a003da51-kube-api-access-4qrx2\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:58:28.835208 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:28.835221    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/53028cc2-acf8-4064-81d6-76c1a003da51-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:58:29.518611 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:58:29.518581    2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53028cc2_acf8_4064_81d6_76c1a003da51.slice/crio-d3b9c0defa069574b7fddcaefa635d180ed9490461c3c677ce66095649362f7c\": RecentStats: unable to find data in memory cache]"
Jul 29 10:58:29.533116 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:29.533089    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"
Jul 29 10:58:29.533514 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:29.533082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c" event={"ID":"53028cc2-acf8-4064-81d6-76c1a003da51","Type":"ContainerDied","Data":"d3b9c0defa069574b7fddcaefa635d180ed9490461c3c677ce66095649362f7c"}
Jul 29 10:58:29.533514 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:29.533235    2562 scope.go:117] "RemoveContainer" containerID="7c9e5706c588c38415c6cf9ffcca807ea513a681a9e2400648e7287c524db1b6"
Jul 29 10:58:29.541878 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:29.541858    2562 scope.go:117] "RemoveContainer" containerID="5c17a964ee9e451d69a247681ccdf0e8f62c5c46a52546391610ed6337007a0c"
Jul 29 10:58:29.559517 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:29.559469    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"]
Jul 29 10:58:29.565413 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:29.565384    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d2d23-predictor-658b5fdc7-kx72c"]
Jul 29 10:58:31.507725 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:31.507692    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" path="/var/lib/kubelet/pods/53028cc2-acf8-4064-81d6-76c1a003da51/volumes"
Jul 29 10:58:33.532422 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:33.532393    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:58:33.532888 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:33.532862    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 29 10:58:43.532877 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:43.532824    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 29 10:58:53.533819 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:58:53.533770    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 29 10:59:03.533480 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:03.533434    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused"
Jul 29 10:59:13.534579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:13.534551    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:59:24.783983 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:24.783945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:24.927034 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:24.926996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:24.938666 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:24.938634    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"]
Jul 29 10:59:24.938890 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:24.938863    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" containerID="cri-o://4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4" gracePeriod=30
Jul 29 10:59:24.939000 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:24.938900    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kube-rbac-proxy" containerID="cri-o://f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b" gracePeriod=30
Jul 29 10:59:24.988247 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:24.988213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:25.030280 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030242    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"]
Jul 29 10:59:25.030673 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030660    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container"
Jul 29 10:59:25.030748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030675    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container"
Jul 29 10:59:25.030748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030707    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kube-rbac-proxy"
Jul 29 10:59:25.030748 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030714    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kube-rbac-proxy"
Jul 29 10:59:25.030841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030773    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kube-rbac-proxy"
Jul 29 10:59:25.030841 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.030786    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="53028cc2-acf8-4064-81d6-76c1a003da51" containerName="kserve-container"
Jul 29 10:59:25.034432 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.034375    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.037060 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.037041    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-965a2-kube-rbac-proxy-sar-config\""
Jul 29 10:59:25.037157 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.037138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-965a2-predictor-serving-cert\""
Jul 29 10:59:25.043552 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.043507    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"]
Jul 29 10:59:25.050790 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.050744    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:25.113033 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.113001    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:25.140228 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.140181    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk72l\" (UniqueName: \"kubernetes.io/projected/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-api-access-sk72l\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.140423 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.140294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.140423 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.140369    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.241573 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.241533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk72l\" (UniqueName: \"kubernetes.io/projected/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-api-access-sk72l\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.241780 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.241631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.241780 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.241671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.241906 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:59:25.241790    2562 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-965a2-predictor-serving-cert: secret "error-404-isvc-965a2-predictor-serving-cert" not found
Jul 29 10:59:25.241906 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:59:25.241861    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls podName:83a6e9dc-18be-4c29-a7be-abccce72a78c nodeName:}" failed. No retries permitted until 2026-07-29 10:59:25.741840707 +0000 UTC m=+1878.730690157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls") pod "error-404-isvc-965a2-predictor-6df948c77-gjp88" (UID: "83a6e9dc-18be-4c29-a7be-abccce72a78c") : secret "error-404-isvc-965a2-predictor-serving-cert" not found
Jul 29 10:59:25.242426 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.242404    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.252446 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.252413    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk72l\" (UniqueName: \"kubernetes.io/projected/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-api-access-sk72l\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.309648 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.309568    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:25.410377 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.410346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:25.746752 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.746712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.749135 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.749105    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls\") pod \"error-404-isvc-965a2-predictor-6df948c77-gjp88\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") " pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:25.823281 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.823245    2562 generic.go:356] "Generic (PLEG): container finished" podID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerID="f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b" exitCode=2
Jul 29 10:59:25.823692 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.823316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" event={"ID":"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc","Type":"ContainerDied","Data":"f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b"}
Jul 29 10:59:25.947195 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:25.947152    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:26.089826 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.089800    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"]
Jul 29 10:59:26.092614 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:59:26.092583    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a6e9dc_18be_4c29_a7be_abccce72a78c.slice/crio-9f89e42daf7347cbad069b1ce94d5df2cab5e037dfbb6abb676a3e0a7a3e23ef WatchSource:0}: Error finding container 9f89e42daf7347cbad069b1ce94d5df2cab5e037dfbb6abb676a3e0a7a3e23ef: Status 404 returned error can't find the container with id 9f89e42daf7347cbad069b1ce94d5df2cab5e037dfbb6abb676a3e0a7a3e23ef
Jul 29 10:59:26.829799 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.829758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" event={"ID":"83a6e9dc-18be-4c29-a7be-abccce72a78c","Type":"ContainerStarted","Data":"037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43"}
Jul 29 10:59:26.830288 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.829953    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:26.830288 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.829996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" event={"ID":"83a6e9dc-18be-4c29-a7be-abccce72a78c","Type":"ContainerStarted","Data":"2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d"}
Jul 29 10:59:26.830288 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.830022    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:26.830288 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.830044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" event={"ID":"83a6e9dc-18be-4c29-a7be-abccce72a78c","Type":"ContainerStarted","Data":"9f89e42daf7347cbad069b1ce94d5df2cab5e037dfbb6abb676a3e0a7a3e23ef"}
Jul 29 10:59:26.831529 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.831501    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 10:59:26.849994 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:26.849931    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podStartSLOduration=1.849913572 podStartE2EDuration="1.849913572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:59:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:59:26.847687418 +0000 UTC m=+1879.836536903" watchObservedRunningTime="2026-07-29 10:59:26.849913572 +0000 UTC m=+1879.838763028"
Jul 29 10:59:27.112465 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:27.112371    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.41:8643/healthz\": dial tcp 10.132.0.41:8643: connect: connection refused"
Jul 29 10:59:27.117720 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:27.117686    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 29 10:59:27.835129 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:27.835090    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 10:59:28.391679 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.391649    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:59:28.472630 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.472531    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-proxy-tls\") pod \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") "
Jul 29 10:59:28.472630 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.472584    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6d5k\" (UniqueName: \"kubernetes.io/projected/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-api-access-c6d5k\") pod \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") "
Jul 29 10:59:28.472872 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.472662    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-rbac-proxy-sar-config\") pod \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\" (UID: \"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc\") "
Jul 29 10:59:28.473095 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.473064    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" (UID: "ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:59:28.474620 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.474593    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" (UID: "ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:59:28.474740 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.474699    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-api-access-c6d5k" (OuterVolumeSpecName: "kube-api-access-c6d5k") pod "ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" (UID: "ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc"). InnerVolumeSpecName "kube-api-access-c6d5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:59:28.573819 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.573775    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:59:28.573819 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.573808    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:59:28.573819 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.573821    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6d5k\" (UniqueName: \"kubernetes.io/projected/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc-kube-api-access-c6d5k\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:59:28.842114 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.842069    2562 generic.go:356] "Generic (PLEG): container finished" podID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerID="4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4" exitCode=0
Jul 29 10:59:28.842114 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.842116    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" event={"ID":"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc","Type":"ContainerDied","Data":"4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4"}
Jul 29 10:59:28.842617 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.842123    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"
Jul 29 10:59:28.842617 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.842179    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs" event={"ID":"ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc","Type":"ContainerDied","Data":"eb69c3d99192e96402d7887ad3ddf1cae518f33e79a75955c0c43814c4d0074e"}
Jul 29 10:59:28.842617 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.842214    2562 scope.go:117] "RemoveContainer" containerID="f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b"
Jul 29 10:59:28.851866 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.851844    2562 scope.go:117] "RemoveContainer" containerID="4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4"
Jul 29 10:59:28.860361 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.860336    2562 scope.go:117] "RemoveContainer" containerID="f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b"
Jul 29 10:59:28.860722 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:59:28.860700    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b\": container with ID starting with f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b not found: ID does not exist" containerID="f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b"
Jul 29 10:59:28.860789 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.860734    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b"} err="failed to get container status \"f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b\": rpc error: code = NotFound desc = could not find container \"f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b\": container with ID starting with f5cf2a095dbd9683a758cc69be79cb5d1ac20ad310fa8b3ba603363d68c5268b not found: ID does not exist"
Jul 29 10:59:28.860789 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.860753    2562 scope.go:117] "RemoveContainer" containerID="4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4"
Jul 29 10:59:28.861010 ip-10-0-143-36 kubenswrapper[2562]: E0729 10:59:28.860993    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4\": container with ID starting with 4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4 not found: ID does not exist" containerID="4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4"
Jul 29 10:59:28.861052 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.861016    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4"} err="failed to get container status \"4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4\": rpc error: code = NotFound desc = could not find container \"4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4\": container with ID starting with 4c457e6f7f0076f1d0eeaafee8a4386f8ab75acac9ef437ec56a49b247e1eda4 not found: ID does not exist"
Jul 29 10:59:28.866118 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.866083    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"]
Jul 29 10:59:28.872086 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:28.872049    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0934b-predictor-5949bbfc75-kftfs"]
Jul 29 10:59:29.508798 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:29.508762    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" path="/var/lib/kubelet/pods/ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc/volumes"
Jul 29 10:59:32.839404 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:32.839375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 10:59:32.839860 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:32.839838    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 10:59:35.185954 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.185872    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:35.272431 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.272381    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"]
Jul 29 10:59:35.272692 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.272664    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container" containerID="cri-o://32aae38b912eac31428200207be3ed51f08fc33666a6503c8f880d6b92265a21" gracePeriod=30
Jul 29 10:59:35.272767 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.272713    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kube-rbac-proxy" containerID="cri-o://feb8a433e344b7ad62a8422471259964f52427c02a0a9062967bb7311fb146b9" gracePeriod=30
Jul 29 10:59:35.292850 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.292821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:35.354967 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.354938    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:35.434882 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.434851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:35.449197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449109    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"]
Jul 29 10:59:35.449567 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449549    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container"
Jul 29 10:59:35.449567 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449568    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container"
Jul 29 10:59:35.449698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449587    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kube-rbac-proxy"
Jul 29 10:59:35.449698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449593    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kube-rbac-proxy"
Jul 29 10:59:35.449698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449666    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kserve-container"
Jul 29 10:59:35.449698 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.449676    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee8afd5e-7b5f-4b67-b55b-fb8f8dfccbfc" containerName="kube-rbac-proxy"
Jul 29 10:59:35.452738 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.452719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.455240 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.455222    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-afa2d-predictor-serving-cert\""
Jul 29 10:59:35.455343 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.455248    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-afa2d-kube-rbac-proxy-sar-config\""
Jul 29 10:59:35.466611 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.466575    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"]
Jul 29 10:59:35.606365 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.606308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:35.638579 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.638535    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpw8m\" (UniqueName: \"kubernetes.io/projected/26b8799d-090d-443c-a60c-ee2419adef4e-kube-api-access-qpw8m\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.638758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.638601    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/26b8799d-090d-443c-a60c-ee2419adef4e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.638758 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.638697    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26b8799d-090d-443c-a60c-ee2419adef4e-proxy-tls\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.739582 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.739543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpw8m\" (UniqueName: \"kubernetes.io/projected/26b8799d-090d-443c-a60c-ee2419adef4e-kube-api-access-qpw8m\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.739750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.739612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/26b8799d-090d-443c-a60c-ee2419adef4e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.739750 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.739690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26b8799d-090d-443c-a60c-ee2419adef4e-proxy-tls\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.740270 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.740250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/26b8799d-090d-443c-a60c-ee2419adef4e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.742109 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.742091    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26b8799d-090d-443c-a60c-ee2419adef4e-proxy-tls\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.748800 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.748776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpw8m\" (UniqueName: \"kubernetes.io/projected/26b8799d-090d-443c-a60c-ee2419adef4e-kube-api-access-qpw8m\") pod \"error-404-isvc-afa2d-predictor-75594848c5-cc7sx\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") " pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.765695 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.765643    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:35.882815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.882778    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerID="feb8a433e344b7ad62a8422471259964f52427c02a0a9062967bb7311fb146b9" exitCode=2
Jul 29 10:59:35.882815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.882823    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" event={"ID":"5b77336e-7fa8-40fa-8886-cbf3b2b9b729","Type":"ContainerDied","Data":"feb8a433e344b7ad62a8422471259964f52427c02a0a9062967bb7311fb146b9"}
Jul 29 10:59:35.905200 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:35.905173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"]
Jul 29 10:59:35.907358 ip-10-0-143-36 kubenswrapper[2562]: W0729 10:59:35.907291    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b8799d_090d_443c_a60c_ee2419adef4e.slice/crio-2df6be54db5ca71f3f462833443aa47bc3829f9ccb14ea6326f78b1b4826687a WatchSource:0}: Error finding container 2df6be54db5ca71f3f462833443aa47bc3829f9ccb14ea6326f78b1b4826687a: Status 404 returned error can't find the container with id 2df6be54db5ca71f3f462833443aa47bc3829f9ccb14ea6326f78b1b4826687a
Jul 29 10:59:36.105585 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:36.105549    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 10:59:36.889771 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:36.889729    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" event={"ID":"26b8799d-090d-443c-a60c-ee2419adef4e","Type":"ContainerStarted","Data":"23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2"}
Jul 29 10:59:36.890197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:36.889804    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" event={"ID":"26b8799d-090d-443c-a60c-ee2419adef4e","Type":"ContainerStarted","Data":"8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f"}
Jul 29 10:59:36.890197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:36.889834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" event={"ID":"26b8799d-090d-443c-a60c-ee2419adef4e","Type":"ContainerStarted","Data":"2df6be54db5ca71f3f462833443aa47bc3829f9ccb14ea6326f78b1b4826687a"}
Jul 29 10:59:36.890197 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:36.889867    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:36.910561 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:36.910508    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podStartSLOduration=1.910490059 podStartE2EDuration="1.910490059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 10:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 10:59:36.908821702 +0000 UTC m=+1889.897671157" watchObservedRunningTime="2026-07-29 10:59:36.910490059 +0000 UTC m=+1889.899339514"
Jul 29 10:59:37.897777 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:37.897735    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:37.899246 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:37.899207    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 29 10:59:38.528331 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.528270    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.42:8643/healthz\": dial tcp 10.132.0.42:8643: connect: connection refused"
Jul 29 10:59:38.905215 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.905187    2562 generic.go:356] "Generic (PLEG): container finished" podID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerID="32aae38b912eac31428200207be3ed51f08fc33666a6503c8f880d6b92265a21" exitCode=0
Jul 29 10:59:38.905590 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.905254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" event={"ID":"5b77336e-7fa8-40fa-8886-cbf3b2b9b729","Type":"ContainerDied","Data":"32aae38b912eac31428200207be3ed51f08fc33666a6503c8f880d6b92265a21"}
Jul 29 10:59:38.905672 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.905646    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 29 10:59:38.924845 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.924816    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:59:38.967482 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.967448    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-proxy-tls\") pod \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") "
Jul 29 10:59:38.967659 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.967509    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xndzz\" (UniqueName: \"kubernetes.io/projected/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-api-access-xndzz\") pod \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") "
Jul 29 10:59:38.967659 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.967554    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-rbac-proxy-sar-config\") pod \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\" (UID: \"5b77336e-7fa8-40fa-8886-cbf3b2b9b729\") "
Jul 29 10:59:38.968028 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.967996    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5b77336e-7fa8-40fa-8886-cbf3b2b9b729" (UID: "5b77336e-7fa8-40fa-8886-cbf3b2b9b729"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 10:59:38.969761 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.969740    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5b77336e-7fa8-40fa-8886-cbf3b2b9b729" (UID: "5b77336e-7fa8-40fa-8886-cbf3b2b9b729"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 10:59:38.969855 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:38.969776    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-api-access-xndzz" (OuterVolumeSpecName: "kube-api-access-xndzz") pod "5b77336e-7fa8-40fa-8886-cbf3b2b9b729" (UID: "5b77336e-7fa8-40fa-8886-cbf3b2b9b729"). InnerVolumeSpecName "kube-api-access-xndzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 10:59:39.068559 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.068444    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xndzz\" (UniqueName: \"kubernetes.io/projected/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-api-access-xndzz\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.068559 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.068491    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.068559 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.068504    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b77336e-7fa8-40fa-8886-cbf3b2b9b729-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 10:59:39.912553 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.912473    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq" event={"ID":"5b77336e-7fa8-40fa-8886-cbf3b2b9b729","Type":"ContainerDied","Data":"f8b2f7f753c012febb430b3fe15ee2a95ddb34b500895efa1e8a394d6e3ef8d1"}
Jul 29 10:59:39.912553 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.912500    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"
Jul 29 10:59:39.912553 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.912542    2562 scope.go:117] "RemoveContainer" containerID="feb8a433e344b7ad62a8422471259964f52427c02a0a9062967bb7311fb146b9"
Jul 29 10:59:39.921848 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.921828    2562 scope.go:117] "RemoveContainer" containerID="32aae38b912eac31428200207be3ed51f08fc33666a6503c8f880d6b92265a21"
Jul 29 10:59:39.932086 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.932057    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"]
Jul 29 10:59:39.933744 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:39.933724    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f5066-predictor-645f7d7b85-fsdjq"]
Jul 29 10:59:41.507815 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:41.507777    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" path="/var/lib/kubelet/pods/5b77336e-7fa8-40fa-8886-cbf3b2b9b729/volumes"
Jul 29 10:59:42.839830 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:42.839787    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 10:59:43.910571 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:43.910540    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 10:59:43.911062 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:43.911029    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 29 10:59:52.840020 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:52.839976    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 10:59:53.911580 ip-10-0-143-36 kubenswrapper[2562]: I0729 10:59:53.911534    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 29 11:00:02.839849 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:02.839803    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 11:00:03.911298 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:03.911257    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 29 11:00:12.840876 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:12.840825    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.43:8080: connect: connection refused"
Jul 29 11:00:13.911028 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:13.910983    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused"
Jul 29 11:00:22.840489 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:22.840457    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 11:00:23.912099 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:23.912069    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 11:00:25.012346 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:25.012285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:00:27.010503 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:27.010470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:00:35.437651 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:35.437614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:00:35.683024 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:00:35.682987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:03:07.657993 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:03:07.657967    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:03:07.664119 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:03:07.664096    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:03:23.324989 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:03:23.324944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:03:23.414636 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:03:23.414602    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:04:25.814239 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:04:25.814146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:04:32.763502 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:04:32.763465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:05:32.711072 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:05:32.710980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:08:07.688396 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:07.688366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:08:07.695808 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:07.695776    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:08:49.983579 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:49.983546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:08:50.094116 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.094086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:08:50.114597 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.114556    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"]
Jul 29 11:08:50.114939 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.114907    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container" containerID="cri-o://8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f" gracePeriod=30
Jul 29 11:08:50.114939 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.114921    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kube-rbac-proxy" containerID="cri-o://23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2" gracePeriod=30
Jul 29 11:08:50.170640 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.170612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:08:50.613570 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.613538    2562 generic.go:356] "Generic (PLEG): container finished" podID="26b8799d-090d-443c-a60c-ee2419adef4e" containerID="23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2" exitCode=2
Jul 29 11:08:50.613760 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:50.613612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" event={"ID":"26b8799d-090d-443c-a60c-ee2419adef4e","Type":"ContainerDied","Data":"23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2"}
Jul 29 11:08:53.573520 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.573490    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 11:08:53.637723 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.637688    2562 generic.go:356] "Generic (PLEG): container finished" podID="26b8799d-090d-443c-a60c-ee2419adef4e" containerID="8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f" exitCode=0
Jul 29 11:08:53.637904 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.637740    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"
Jul 29 11:08:53.637904 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.637750    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" event={"ID":"26b8799d-090d-443c-a60c-ee2419adef4e","Type":"ContainerDied","Data":"8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f"}
Jul 29 11:08:53.637904 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.637806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx" event={"ID":"26b8799d-090d-443c-a60c-ee2419adef4e","Type":"ContainerDied","Data":"2df6be54db5ca71f3f462833443aa47bc3829f9ccb14ea6326f78b1b4826687a"}
Jul 29 11:08:53.637904 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.637836    2562 scope.go:117] "RemoveContainer" containerID="23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2"
Jul 29 11:08:53.646715 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.646692    2562 scope.go:117] "RemoveContainer" containerID="8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f"
Jul 29 11:08:53.648994 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.648949    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpw8m\" (UniqueName: \"kubernetes.io/projected/26b8799d-090d-443c-a60c-ee2419adef4e-kube-api-access-qpw8m\") pod \"26b8799d-090d-443c-a60c-ee2419adef4e\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") "
Jul 29 11:08:53.649080 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.648994    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26b8799d-090d-443c-a60c-ee2419adef4e-proxy-tls\") pod \"26b8799d-090d-443c-a60c-ee2419adef4e\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") "
Jul 29 11:08:53.649139 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.649095    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/26b8799d-090d-443c-a60c-ee2419adef4e-kube-rbac-proxy-sar-config\") pod \"26b8799d-090d-443c-a60c-ee2419adef4e\" (UID: \"26b8799d-090d-443c-a60c-ee2419adef4e\") "
Jul 29 11:08:53.649529 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.649496    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b8799d-090d-443c-a60c-ee2419adef4e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "26b8799d-090d-443c-a60c-ee2419adef4e" (UID: "26b8799d-090d-443c-a60c-ee2419adef4e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 11:08:53.651113 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.651083    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26b8799d-090d-443c-a60c-ee2419adef4e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "26b8799d-090d-443c-a60c-ee2419adef4e" (UID: "26b8799d-090d-443c-a60c-ee2419adef4e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 11:08:53.651204 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.651118    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b8799d-090d-443c-a60c-ee2419adef4e-kube-api-access-qpw8m" (OuterVolumeSpecName: "kube-api-access-qpw8m") pod "26b8799d-090d-443c-a60c-ee2419adef4e" (UID: "26b8799d-090d-443c-a60c-ee2419adef4e"). InnerVolumeSpecName "kube-api-access-qpw8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 11:08:53.658052 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.657956    2562 scope.go:117] "RemoveContainer" containerID="23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2"
Jul 29 11:08:53.658355 ip-10-0-143-36 kubenswrapper[2562]: E0729 11:08:53.658334    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2\": container with ID starting with 23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2 not found: ID does not exist" containerID="23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2"
Jul 29 11:08:53.658415 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.658366    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2"} err="failed to get container status \"23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2\": rpc error: code = NotFound desc = could not find container \"23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2\": container with ID starting with 23a82440fd990e2d418b49114e4eef1a47bd7085ce574ec303f4377e23dcbfa2 not found: ID does not exist"
Jul 29 11:08:53.658415 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.658392    2562 scope.go:117] "RemoveContainer" containerID="8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f"
Jul 29 11:08:53.658743 ip-10-0-143-36 kubenswrapper[2562]: E0729 11:08:53.658718    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f\": container with ID starting with 8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f not found: ID does not exist" containerID="8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f"
Jul 29 11:08:53.658847 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.658750    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f"} err="failed to get container status \"8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f\": rpc error: code = NotFound desc = could not find container \"8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f\": container with ID starting with 8c511cb3cebc87f07ee73bd79e90c78f8237f0cb6380658b2fd6d14a5c391d3f not found: ID does not exist"
Jul 29 11:08:53.750364 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.750297    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/26b8799d-090d-443c-a60c-ee2419adef4e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 11:08:53.750364 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.750359    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpw8m\" (UniqueName: \"kubernetes.io/projected/26b8799d-090d-443c-a60c-ee2419adef4e-kube-api-access-qpw8m\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 11:08:53.750364 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.750368    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/26b8799d-090d-443c-a60c-ee2419adef4e-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 11:08:53.961247 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.961211    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"]
Jul 29 11:08:53.964840 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:53.964797    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-afa2d-predictor-75594848c5-cc7sx"]
Jul 29 11:08:55.507508 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:08:55.507476    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" path="/var/lib/kubelet/pods/26b8799d-090d-443c-a60c-ee2419adef4e/volumes"
Jul 29 11:11:49.087334 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:11:49.087271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:13:07.721992 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:13:07.721853    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:13:07.734478 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:13:07.734447    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:16:44.425754 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.425717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:16:44.563283 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.563244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:16:44.597091 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.597056    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"]
Jul 29 11:16:44.597368 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.597309    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container" containerID="cri-o://2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d" gracePeriod=30
Jul 29 11:16:44.597368 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.597354    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kube-rbac-proxy" containerID="cri-o://037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43" gracePeriod=30
Jul 29 11:16:44.629028 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.628994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:16:44.870910 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.870876    2562 generic.go:356] "Generic (PLEG): container finished" podID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerID="037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43" exitCode=2
Jul 29 11:16:44.871087 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:44.870959    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" event={"ID":"83a6e9dc-18be-4c29-a7be-abccce72a78c","Type":"ContainerDied","Data":"037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43"}
Jul 29 11:16:46.157208 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:46.157169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:16:46.248147 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:46.248109    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:16:47.835764 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:47.835713    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.43:8643/healthz\": dial tcp 10.132.0.43:8643: connect: connection refused"
Jul 29 11:16:48.159623 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.159593    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 11:16:48.251413 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.251381    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-rbac-proxy-sar-config\") pod \"83a6e9dc-18be-4c29-a7be-abccce72a78c\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") "
Jul 29 11:16:48.251607 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.251458    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls\") pod \"83a6e9dc-18be-4c29-a7be-abccce72a78c\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") "
Jul 29 11:16:48.251607 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.251484    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk72l\" (UniqueName: \"kubernetes.io/projected/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-api-access-sk72l\") pod \"83a6e9dc-18be-4c29-a7be-abccce72a78c\" (UID: \"83a6e9dc-18be-4c29-a7be-abccce72a78c\") "
Jul 29 11:16:48.251835 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.251807    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "83a6e9dc-18be-4c29-a7be-abccce72a78c" (UID: "83a6e9dc-18be-4c29-a7be-abccce72a78c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 11:16:48.253718 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.253686    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-api-access-sk72l" (OuterVolumeSpecName: "kube-api-access-sk72l") pod "83a6e9dc-18be-4c29-a7be-abccce72a78c" (UID: "83a6e9dc-18be-4c29-a7be-abccce72a78c"). InnerVolumeSpecName "kube-api-access-sk72l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 11:16:48.253837 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.253754    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "83a6e9dc-18be-4c29-a7be-abccce72a78c" (UID: "83a6e9dc-18be-4c29-a7be-abccce72a78c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 11:16:48.352500 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.352464    2562 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/83a6e9dc-18be-4c29-a7be-abccce72a78c-proxy-tls\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 11:16:48.352500 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.352493    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk72l\" (UniqueName: \"kubernetes.io/projected/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-api-access-sk72l\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 11:16:48.352500 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.352503    2562 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/83a6e9dc-18be-4c29-a7be-abccce72a78c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-36.ec2.internal\" DevicePath \"\""
Jul 29 11:16:48.892672 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.892641    2562 generic.go:356] "Generic (PLEG): container finished" podID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerID="2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d" exitCode=0
Jul 29 11:16:48.893099 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.892692    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"
Jul 29 11:16:48.893099 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.892718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" event={"ID":"83a6e9dc-18be-4c29-a7be-abccce72a78c","Type":"ContainerDied","Data":"2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d"}
Jul 29 11:16:48.893099 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.892761    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88" event={"ID":"83a6e9dc-18be-4c29-a7be-abccce72a78c","Type":"ContainerDied","Data":"9f89e42daf7347cbad069b1ce94d5df2cab5e037dfbb6abb676a3e0a7a3e23ef"}
Jul 29 11:16:48.893099 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.892785    2562 scope.go:117] "RemoveContainer" containerID="037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43"
Jul 29 11:16:48.902136 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.902115    2562 scope.go:117] "RemoveContainer" containerID="2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d"
Jul 29 11:16:48.910419 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.910393    2562 scope.go:117] "RemoveContainer" containerID="037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43"
Jul 29 11:16:48.910747 ip-10-0-143-36 kubenswrapper[2562]: E0729 11:16:48.910722    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43\": container with ID starting with 037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43 not found: ID does not exist" containerID="037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43"
Jul 29 11:16:48.910858 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.910758    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43"} err="failed to get container status \"037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43\": rpc error: code = NotFound desc = could not find container \"037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43\": container with ID starting with 037e13d6d234548b0a064b5863bb3009bf418da7ba20772b47bb247491277f43 not found: ID does not exist"
Jul 29 11:16:48.910858 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.910786    2562 scope.go:117] "RemoveContainer" containerID="2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d"
Jul 29 11:16:48.911051 ip-10-0-143-36 kubenswrapper[2562]: E0729 11:16:48.911034    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d\": container with ID starting with 2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d not found: ID does not exist" containerID="2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d"
Jul 29 11:16:48.911091 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.911059    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d"} err="failed to get container status \"2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d\": rpc error: code = NotFound desc = could not find container \"2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d\": container with ID starting with 2489897f8dfecb9e32ac2d5dfd7b45ec75ca1334995f7d9a8905faae01f2031d not found: ID does not exist"
Jul 29 11:16:48.916206 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.916174    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"]
Jul 29 11:16:48.920613 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:48.920584    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-965a2-predictor-6df948c77-gjp88"]
Jul 29 11:16:49.507373 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:16:49.507337    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" path="/var/lib/kubelet/pods/83a6e9dc-18be-4c29-a7be-abccce72a78c/volumes"
Jul 29 11:17:17.581823 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:17.581778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:17:18.803311 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:18.803270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:17:22.075539 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:22.075505    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xp6pm_198702a2-64b9-4023-84f4-cb0388b57f62/global-pull-secret-syncer/0.log"
Jul 29 11:17:22.140528 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:22.140488    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log"
Jul 29 11:17:22.311093 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:22.311051    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-36.ec2.internal_dba68a4d6621fead7294c30802645215/haproxy/0.log"
Jul 29 11:17:23.688234 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:23.688198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:17:23.791711 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:23.791676    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dldrq_8c955947-80b8-42b8-8900-db6cc6c15b89/konnectivity-agent/0.log" limit=1024
Jul 29 11:17:26.065673 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.065647    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/alertmanager/0.log"
Jul 29 11:17:26.101002 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.100970    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/config-reloader/0.log"
Jul 29 11:17:26.132100 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.132072    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/kube-rbac-proxy-web/0.log"
Jul 29 11:17:26.160939 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.160901    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/kube-rbac-proxy/0.log"
Jul 29 11:17:26.191395 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.191363    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/kube-rbac-proxy-metric/0.log"
Jul 29 11:17:26.219935 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.219903    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/prom-label-proxy/0.log"
Jul 29 11:17:26.252845 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.252813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4c5072e6-2a3f-42f3-a41f-ac89a4544ec2/init-config-reloader/0.log"
Jul 29 11:17:26.514302 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.514265    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rnvrg_f72d1adb-1bb4-4a92-afb3-362b0aa25aac/kube-state-metrics/0.log"
Jul 29 11:17:26.541225 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.541186    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rnvrg_f72d1adb-1bb4-4a92-afb3-362b0aa25aac/kube-rbac-proxy-main/0.log"
Jul 29 11:17:26.570118 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.570088    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rnvrg_f72d1adb-1bb4-4a92-afb3-362b0aa25aac/kube-rbac-proxy-self/0.log"
Jul 29 11:17:26.604333 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.604288    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-fdd69bcfc-f9mxl_6669f13e-d4be-4e7a-b246-5c1f0b2dc8aa/metrics-server/0.log"
Jul 29 11:17:26.638478 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.638449    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-4wjsj_5c8e26e8-735b-4b82-9174-fafed08413a0/monitoring-plugin/0.log"
Jul 29 11:17:26.878602 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.878517    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk5zv_7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89/node-exporter/0.log"
Jul 29 11:17:26.905615 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.905581    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk5zv_7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89/kube-rbac-proxy/0.log"
Jul 29 11:17:26.935559 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:26.935531    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hk5zv_7b7db9ff-b7bd-4f36-9800-ef92b8dd4f89/init-textfile/0.log"
Jul 29 11:17:27.375462 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.375426    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-88b5d58c7-g888w_517d370f-445f-464f-bf88-aa374d8852db/telemeter-client/0.log"
Jul 29 11:17:27.401011 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.400978    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-88b5d58c7-g888w_517d370f-445f-464f-bf88-aa374d8852db/reload/0.log"
Jul 29 11:17:27.428270 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.428238    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-88b5d58c7-g888w_517d370f-445f-464f-bf88-aa374d8852db/kube-rbac-proxy/0.log"
Jul 29 11:17:27.463827 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.463793    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f8f5c957-zpks8_bfca97cd-7242-42f8-9554-38467634e9a4/thanos-query/0.log"
Jul 29 11:17:27.489622 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.489595    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f8f5c957-zpks8_bfca97cd-7242-42f8-9554-38467634e9a4/kube-rbac-proxy-web/0.log"
Jul 29 11:17:27.515843 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.515813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f8f5c957-zpks8_bfca97cd-7242-42f8-9554-38467634e9a4/kube-rbac-proxy/0.log"
Jul 29 11:17:27.543944 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.543914    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f8f5c957-zpks8_bfca97cd-7242-42f8-9554-38467634e9a4/prom-label-proxy/0.log"
Jul 29 11:17:27.571072 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.571042    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f8f5c957-zpks8_bfca97cd-7242-42f8-9554-38467634e9a4/kube-rbac-proxy-rules/0.log"
Jul 29 11:17:27.601881 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:27.601851    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64f8f5c957-zpks8_bfca97cd-7242-42f8-9554-38467634e9a4/kube-rbac-proxy-metrics/0.log"
Jul 29 11:17:29.510626 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:29.510594    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/1.log"
Jul 29 11:17:29.519114 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:29.519085    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-sb872_7ee7cb80-9446-46f9-ad0b-e21752077bea/console-operator/2.log"
Jul 29 11:17:30.000729 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.000695    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8555ffc549-whvmc_3121acfc-b574-4032-8b77-47dff9312eb8/console/0.log"
Jul 29 11:17:30.045743 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.045711    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-hjpt2_f68b5087-14ea-4fa6-9ff6-3b5870681402/download-server/0.log"
Jul 29 11:17:30.577922 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.577886    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"]
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578238    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578249    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578259    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578264    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578272    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578278    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578287    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578292    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578311    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578316    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578343    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container"
Jul 29 11:17:30.578347 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578351    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container"
Jul 29 11:17:30.578758 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578404    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578758 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578413    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b77336e-7fa8-40fa-8886-cbf3b2b9b729" containerName="kserve-container"
Jul 29 11:17:30.578758 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578421    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kserve-container"
Jul 29 11:17:30.578758 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578427    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kserve-container"
Jul 29 11:17:30.578758 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578432    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b8799d-090d-443c-a60c-ee2419adef4e" containerName="kube-rbac-proxy"
Jul 29 11:17:30.578758 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.578439    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a6e9dc-18be-4c29-a7be-abccce72a78c" containerName="kube-rbac-proxy"
Jul 29 11:17:30.581600 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.581576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.584175 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.584153    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8kqzn\"/\"default-dockercfg-9wrc4\""
Jul 29 11:17:30.584692 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.584670    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8kqzn\"/\"openshift-service-ca.crt\""
Jul 29 11:17:30.585602 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.585582    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8kqzn\"/\"kube-root-ca.crt\""
Jul 29 11:17:30.591405 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.591378    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"]
Jul 29 11:17:30.642516 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.642481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-proc\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.642751 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.642554    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkrsb\" (UniqueName: \"kubernetes.io/projected/e9ae2b31-ea53-44c7-887f-4103c3c17420-kube-api-access-gkrsb\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.642751 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.642607    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-4td4l_0ba8dc12-e5c0-4be8-bd18-54dcbdfd28bb/volume-data-source-validator/0.log"
Jul 29 11:17:30.642751 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.642618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-lib-modules\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.642751 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.642649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-podres\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.642751 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.642701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-sys\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743698 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-proc\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743877 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkrsb\" (UniqueName: \"kubernetes.io/projected/e9ae2b31-ea53-44c7-887f-4103c3c17420-kube-api-access-gkrsb\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743877 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-lib-modules\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743877 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-podres\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743877 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743775    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-proc\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743877 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-sys\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.743877 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743836    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-sys\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.744076 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743896    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-podres\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.744076 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.743923    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9ae2b31-ea53-44c7-887f-4103c3c17420-lib-modules\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.753446 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.753415    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkrsb\" (UniqueName: \"kubernetes.io/projected/e9ae2b31-ea53-44c7-887f-4103c3c17420-kube-api-access-gkrsb\") pod \"perf-node-gather-daemonset-jzffj\" (UID: \"e9ae2b31-ea53-44c7-887f-4103c3c17420\") " pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:30.892465 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:30.892361    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:31.038421 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:31.038196    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"]
Jul 29 11:17:31.041420 ip-10-0-143-36 kubenswrapper[2562]: W0729 11:17:31.041388    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode9ae2b31_ea53_44c7_887f_4103c3c17420.slice/crio-ee3c267de6b1cce6c2728c1c1e0a520494e58c28f33045ee3cbe7e4191cbd869 WatchSource:0}: Error finding container ee3c267de6b1cce6c2728c1c1e0a520494e58c28f33045ee3cbe7e4191cbd869: Status 404 returned error can't find the container with id ee3c267de6b1cce6c2728c1c1e0a520494e58c28f33045ee3cbe7e4191cbd869
Jul 29 11:17:31.098225 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:31.098194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj" event={"ID":"e9ae2b31-ea53-44c7-887f-4103c3c17420","Type":"ContainerStarted","Data":"ee3c267de6b1cce6c2728c1c1e0a520494e58c28f33045ee3cbe7e4191cbd869"}
Jul 29 11:17:31.708443 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:31.708410    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zbcld_22601efc-2958-474f-8590-3725796c9e2c/dns/0.log"
Jul 29 11:17:31.733090 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:31.733056    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zbcld_22601efc-2958-474f-8590-3725796c9e2c/kube-rbac-proxy/0.log"
Jul 29 11:17:31.881305 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:31.881270    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vkshd_c329c2e8-06df-4cad-b6ea-5cff76ef5978/dns-node-resolver/0.log"
Jul 29 11:17:32.109733 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:32.109691    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj" event={"ID":"e9ae2b31-ea53-44c7-887f-4103c3c17420","Type":"ContainerStarted","Data":"de23361f8be1c8e7fc09c3740a4de1eb6f249e2cbe136fb2d573f03e5e992490"}
Jul 29 11:17:32.109918 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:32.109785    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:32.129515 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:32.129464    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj" podStartSLOduration=2.12944879 podStartE2EDuration="2.12944879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 11:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 11:17:32.128874724 +0000 UTC m=+2965.117724180" watchObservedRunningTime="2026-07-29 11:17:32.12944879 +0000 UTC m=+2965.118298246"
Jul 29 11:17:32.426340 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:32.426235    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sx4n7_0a034c05-e7ae-4fd3-b69b-766314822683/node-ca/0.log"
Jul 29 11:17:33.649299 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:33.649266    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r7d54_0b8bb7ea-2050-4f1d-a5ed-1957c71ae978/serve-healthcheck-canary/0.log"
Jul 29 11:17:34.073632 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:34.073592    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-86nmd_c44085eb-1b3d-4591-9baf-d9f38d5f47a2/kube-rbac-proxy/0.log"
Jul 29 11:17:34.098129 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:34.098096    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-86nmd_c44085eb-1b3d-4591-9baf-d9f38d5f47a2/exporter/0.log"
Jul 29 11:17:34.122651 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:34.122615    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-86nmd_c44085eb-1b3d-4591-9baf-d9f38d5f47a2/extractor/0.log"
Jul 29 11:17:36.600407 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:36.600375    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-65f566c74b-m2xzd_a4a91a83-390d-45ea-b57f-4955428dc0a0/manager/0.log"
Jul 29 11:17:36.637575 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:36.637539    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-84fc959947-szcn7_60153e75-8158-4de3-903c-dcd74bcc3013/manager/0.log"
Jul 29 11:17:38.124619 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:38.124591    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8kqzn/perf-node-gather-daemonset-jzffj"
Jul 29 11:17:41.309041 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:41.309009    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-8725r_71b77379-e25b-45ae-af06-10b5868bb9ab/migrator/0.log"
Jul 29 11:17:41.334156 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:41.334127    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-8725r_71b77379-e25b-45ae-af06-10b5868bb9ab/graceful-termination/0.log"
Jul 29 11:17:41.716791 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:41.716706    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss_2caed74b-6436-4709-9330-ede74f2cb0c2/kube-storage-version-migrator-operator/1.log"
Jul 29 11:17:41.718371 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:41.718342    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-r4sss_2caed74b-6436-4709-9330-ede74f2cb0c2/kube-storage-version-migrator-operator/0.log"
Jul 29 11:17:43.085267 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.085237    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/kube-multus-additional-cni-plugins/0.log"
Jul 29 11:17:43.110969 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.110943    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/egress-router-binary-copy/0.log"
Jul 29 11:17:43.136238 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.136211    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/cni-plugins/0.log"
Jul 29 11:17:43.162271 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.162233    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/bond-cni-plugin/0.log"
Jul 29 11:17:43.186817 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.186785    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/routeoverride-cni/0.log"
Jul 29 11:17:43.211853 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.211828    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/whereabouts-cni-bincopy/0.log"
Jul 29 11:17:43.238647 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.238616    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-x8m2t_abd0629b-e500-4831-b288-292c7a8c8c16/whereabouts-cni/0.log"
Jul 29 11:17:43.313288 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.313257    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wn858_35dae8d5-271b-4bac-837b-be006a50d274/kube-multus/0.log"
Jul 29 11:17:43.398383 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.398269    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hrntc_79f1afb4-3c15-4858-8291-48c7603f443a/network-metrics-daemon/0.log"
Jul 29 11:17:43.419489 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:43.419459    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hrntc_79f1afb4-3c15-4858-8291-48c7603f443a/kube-rbac-proxy/0.log"
Jul 29 11:17:44.541854 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.541802    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/ovn-controller/0.log"
Jul 29 11:17:44.563070 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.563044    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/ovn-acl-logging/0.log"
Jul 29 11:17:44.593298 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.593244    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/kube-rbac-proxy-node/0.log"
Jul 29 11:17:44.619620 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.619592    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/kube-rbac-proxy-ovn-metrics/0.log"
Jul 29 11:17:44.640567 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.640522    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/northd/0.log"
Jul 29 11:17:44.664856 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.664818    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/nbdb/0.log"
Jul 29 11:17:44.688887 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.688856    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/sbdb/0.log"
Jul 29 11:17:44.757020 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:44.756987    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4gwff_f22e8537-58dc-4d81-aee3-c978400cdc80/ovnkube-controller/0.log"
Jul 29 11:17:46.058343 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:46.058288    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-mj9l6_a2c68518-a780-4462-947d-f73790788f23/network-check-target-container/0.log"
Jul 29 11:17:47.098214 ip-10-0-143-36 kubenswrapper[2562]: I0729 11:17:47.098128    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-j7jgr_02a171d0-b0c0-4b86-90a2-4efabff92857/iptables-alerter/0.log"