Jul 07 14:06:55.409961 ip-10-0-130-145 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 14:06:55.409970 ip-10-0-130-145 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 14:06:55.409977 ip-10-0-130-145 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 14:06:55.410183 ip-10-0-130-145 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 14:07:05.507515 ip-10-0-130-145 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 14:07:05.507533 ip-10-0-130-145 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ce28aa984fb348aeaa3a1d4e5ab8a524 --
Jul 07 14:09:04.451006 ip-10-0-130-145 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.845927    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852373    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852575    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852579    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852696    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852699    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852746    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852791    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:09:04.857735 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852850    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852855    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852859    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852863    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852867    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852871    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852884    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852891    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852897    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852901    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852905    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852911    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852918    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852943    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.852964    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853168    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853375    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853380    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853383    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:09:04.859993 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853568    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853585    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853588    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853591    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853594    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853708    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853711    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853797    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853801    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853804    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853808    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853933    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853936    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853971    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.853987    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854006    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854009    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854012    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854015    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854017    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:09:04.862547 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854020    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854023    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854081    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854084    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854088    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854091    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854093    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854096    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854099    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854101    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854104    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854106    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854109    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854112    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854116    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854118    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854122    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854124    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854127    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854130    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:09:04.865216 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854132    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854135    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854137    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854140    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854143    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854145    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854149    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854152    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854154    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854157    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854159    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854162    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854164    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854167    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854171    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854174    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854179    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854182    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854184    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:09:04.867198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854188    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854191    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854193    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854196    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854199    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854201    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854204    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854206    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854209    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854211    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854214    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854217    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854219    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.854222    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855586    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855593    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855596    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855599    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855601    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855604    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:09:04.867977 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855606    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855610    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855614    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855617    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855620    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855623    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855626    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855629    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855631    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855636    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855638    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855641    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855643    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855646    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855648    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855651    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855653    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855656    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855658    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:09:04.870167 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855661    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855664    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855666    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855668    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855671    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855674    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855677    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855679    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855682    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855685    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855687    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855699    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855702    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855704    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855707    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855709    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855712    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855715    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855717    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855720    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:09:04.871659 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855722    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855725    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855729    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855731    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855733    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855736    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855738    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855741    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855743    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855746    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855748    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855751    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855753    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855769    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855772    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855776    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855778    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855781    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855785    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:09:04.873085 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855789    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855792    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855795    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855798    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855800    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855803    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855805    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855808    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855810    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855813    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855816    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855818    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855820    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855823    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855825    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855829    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855832    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855834    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855836    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:09:04.873640 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855840    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855843    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855846    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855848    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855851    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855853    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855856    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855858    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855861    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855863    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855866    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855868    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855871    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855874    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855876    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.855880    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856205    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856210    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856213    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856216    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:09:04.874497 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856219    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856221    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856224    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856228    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856231    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856234    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856237    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856240    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856243    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856246    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856248    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856251    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856253    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856256    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856258    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856261    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856263    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856265    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856268    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856270    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:09:04.875538 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856272    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856275    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856278    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856280    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856284    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856286    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856289    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856291    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856294    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856296    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856298    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856301    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856304    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856306    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856308    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856311    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856313    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856316    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856319    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856321    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:09:04.876348 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856323    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856325    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856328    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856330    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856333    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856335    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856338    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856341    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856343    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856346    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856348    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856351    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856353    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856356    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856358    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856360    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856363    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856367    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856369    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856372    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:09:04.876943 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856374    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856377    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856380    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856382    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856384    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856387    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856389    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856392    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856394    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856396    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856399    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856401    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856403    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856405    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856408    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856411    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856413    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856415    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856419    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856423    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:09:04.877736 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856428    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856430    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856433    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856435    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856440    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856443    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856446    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856448    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856451    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856454    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856457    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856461    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856464    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856466    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.856469    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856534    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856540    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856549    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856555    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856561    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856568    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 14:09:04.878324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856572    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856576    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856579    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856581    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856585    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856588    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856591    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856593    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856596    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856599    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856602    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856606    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856610    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856612    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856615    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856619    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856622    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856625    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856628    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856631    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856634    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856637    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856640    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856643    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 14:09:04.878932 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856646    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856649    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856653    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856655    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856658    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856661    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856663    2570 flags.go:64] FLAG: --enable-server="true"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856668    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856672    2570 flags.go:64] FLAG: --event-burst="100"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856675    2570 flags.go:64] FLAG: --event-qps="50"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856678    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856680    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856683    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856687    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856689    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856692    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856695    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856698    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856701    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856705    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856708    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856710    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856713    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856716    2570 flags.go:64] FLAG: --feature-gates=""
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856719    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 14:09:04.879568 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856722    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856725    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856728    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856730    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856733    2570 flags.go:64] FLAG: --help="false"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856736    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-130-145.ec2.internal"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856739    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856742    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856745    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856748    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856751    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856754    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856771    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856774    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856778    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856781    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856784    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856786    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856790    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856792    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856795    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856798    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856800    2570 flags.go:64] FLAG: --lock-file=""
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856803    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 14:09:04.880362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856806    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856808    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856815    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856818    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856820    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856823    2570 flags.go:64] FLAG: --logging-format="text"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856826    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856829    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856831    2570 flags.go:64] FLAG: --manifest-url=""
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856834    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856838    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856841    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856845    2570 flags.go:64] FLAG: --max-pods="110"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856847    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856850    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856861    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856864    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856867    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856870    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856872    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856883    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856886    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856890    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 14:09:04.881286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856893    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856896    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856905    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856908    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856912    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856914    2570 flags.go:64] FLAG: --port="10250"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856917    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856920    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05957de5795e2cef7"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856923    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856925    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856928    2570 flags.go:64] FLAG: --register-node="true"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856932    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856939    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856942    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856945    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856947    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856951    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856954    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856957    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856959    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856962    2570 flags.go:64] FLAG: --runonce="false"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856965    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856968    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856971    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856974    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 14:09:04.882113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856977    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856980    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856983    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856986    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856988    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856991    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856993    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.856997    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857000    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857003    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857006    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857011    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857013    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857016    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857020    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857023    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857025    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857028    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857032    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857035    2570 flags.go:64] FLAG: --v="2"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857038    2570 flags.go:64] FLAG: --version="false"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857042    2570 flags.go:64] FLAG: --vmodule=""
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857045    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857048    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857159    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:09:05.045549 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857163    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:09:04.920210 ip-10-0-130-145 systemd[1]: Started Kubernetes Kubelet.
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857166    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857169    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857172    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857175    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857179    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857182    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857186    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857189    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857191    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857194    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857197    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857200    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857202    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857207    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857209    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857211    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857214    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857216    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857219    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:09:05.068393 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857221    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857224    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857226    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857228    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857231    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857234    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857237    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857239    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857242    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857245    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857247    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857250    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857252    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857255    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857257    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857260    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857262    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857265    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857267    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:09:05.069487 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857270    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857274    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857278    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857281    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857283    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857285    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857288    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857291    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857294    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857296    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857299    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857301    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857303    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857319    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857323    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857326    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857329    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857332    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857336    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:09:05.070454 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857339    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857342    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857344    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857347    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857349    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857352    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857355    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857358    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857361    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857364    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857367    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857370    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857372    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857375    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857378    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857380    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857383    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857385    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857388    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857390    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:09:05.071049 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857394    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857396    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857399    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857401    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857404    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857406    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857409    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857411    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857414    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857416    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857419    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857422    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857425    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857427    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857430    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857432    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857435    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857437    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857440    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:09:05.072037 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.857442    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.857447    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.864345    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.864359    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.864377    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.864382    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864437    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864443    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864445    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864449    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864452    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864454    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864457    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864459    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864462    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:09:05.072604 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864464    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864467    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864469    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864472    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864475    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864477    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864480    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864482    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864485    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864487    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864490    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864492    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864495    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864497    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864500    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864503    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864505    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864510    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864512    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864517    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:09:05.073142 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864521    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864525    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864528    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864531    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864534    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864536    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864539    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864542    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864545    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864547    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864549    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864552    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864555    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864557    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864560    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864562    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864565    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864568    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864570    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:09:05.073720 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864573    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864576    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864578    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864581    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864584    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864586    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864589    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864591    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864594    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864596    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864600    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864603    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864605    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864608    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864610    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864613    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864616    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864618    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864621    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864623    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:09:05.074363 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864626    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864629    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864632    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864635    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864637    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864640    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864642    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864645    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864647    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864650    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864653    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864656    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864659    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864661    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864664    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864666    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864668    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864671    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864673    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:09:05.074973 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864676    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864678    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864681    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864684    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864687    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864690    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864692    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864695    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864697    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864700    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864702    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864705    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.864710    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864833    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:09:05.075553 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864840    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864845    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864848    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864852    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864855    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864858    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864861    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864864    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864866    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864869    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864871    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864874    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864877    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864879    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864882    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864884    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864887    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864889    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864891    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864894    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:09:05.076021 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864896    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864899    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864903    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864905    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864908    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864910    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864913    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864915    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864918    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864920    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864923    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864926    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864929    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864934    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864937    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864940    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864942    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864945    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864947    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864949    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:09:05.076571 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864952    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864955    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864958    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864960    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864964    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864967    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864969    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864971    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864974    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864976    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864979    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864981    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864984    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864986    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864990    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864993    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864995    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.864998    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865000    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865003    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:09:05.077112 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865005    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865008    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865010    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865013    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865015    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865018    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865021    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865024    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865026    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865029    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865031    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865033    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865036    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865038    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865041    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865043    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865045    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865048    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865051    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865053    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:09:05.077669 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865056    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865058    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865061    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865063    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865065    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865068    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865071    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865074    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865076    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865079    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865081    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865084    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865086    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865089    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865091    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865094    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865096    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:09:05.078236 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:04.865101    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.865106    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.865203    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.868549    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.870017    2570 server.go:1019] "Starting client certificate rotation"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.870223    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.870380    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.873106    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.875626    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.883729    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.886052    2570 log.go:25] "Validated CRI v1 image API"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.887015    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 14:09:05.078716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.894806    2570 fs.go:135] Filesystem UUIDs: map[4d2d07da-36ca-4e52-a80d-c7f69c35a06f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cec98d54-f817-4e2c-8ed4-678f6f330311:/dev/nvme0n1p4]
Jul 07 14:09:05.079078 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.894825    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.899228    2570 manager.go:216] Machine: {Timestamp:2026-07-07 14:09:04.898167236 +0000 UTC m=+0.328465588 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200362 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27c2af8985ff8104eb4e56e7e98f87 SystemUUID:ec27c2af-8985-ff81-04eb-4e56e7e98f87 BootID:ce28aa98-4fb3-48ae-aa3a-1d4e5ab8a524 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8d:48:b3:f3:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8d:48:b3:f3:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:ec:5f:b4:b6:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.899329    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.899390    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.902791    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.902813    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-145.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.903455    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.903464    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 14:09:05.079183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.903471    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.904112    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.905130    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.905195    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911067    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911167    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911193    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911202    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911215    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911223    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911232    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911504    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.911523    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.913551    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.915057    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.915084    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916041    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916054    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916060    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916065    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916073    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916079    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916090    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916098    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916107    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916113    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916121    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916126    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916860    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.916867    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.919545    2570 server.go:1268] "Started kubelet"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.923882    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.923941    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.923969    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.924115    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.924152    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-145.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.924192    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-145.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.924988    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 14:09:05.079746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.926215    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.928027    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-145.ec2.internal.18c0072d9c83b639  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-145.ec2.internal,UID:ip-10-0-130-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-145.ec2.internal,},FirstTimestamp:2026-07-07 14:09:04.919516729 +0000 UTC m=+0.349815080,LastTimestamp:2026-07-07 14:09:04.919516729 +0000 UTC m=+0.349815080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-145.ec2.internal,}"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.929309    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.929399    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.929442    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.929921    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.929935    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.930023    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.930140    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.930147    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.930910    2570 factory.go:55] Registering systemd factory
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.930933    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.931044    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.931119    2570 factory.go:153] Registering CRI-O factory
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.931127    2570 factory.go:223] Registration of the crio container factory successfully
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.931176    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.931198    2570 factory.go:103] Registering Raw factory
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.931256    2570 manager.go:1196] Started watching for new ooms in manager
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.931549    2570 manager.go:349] Starting recovery of all containers
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.942402    2570 manager.go:354] Recovery completed
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.942622    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.942629    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-145.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.946713    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.948476    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.948501    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.948519    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949118    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949127    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949131    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949140    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949847    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949871    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.949882    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.080808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.950188    2570 policy_none.go:49] "None policy: Start"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.950200    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.950210    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.950922    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-145.ec2.internal.18c0072d9e3dd40e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-145.ec2.internal,UID:ip-10-0-130-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-145.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-145.ec2.internal,},FirstTimestamp:2026-07-07 14:09:04.948491278 +0000 UTC m=+0.378789633,LastTimestamp:2026-07-07 14:09:04.948491278 +0000 UTC m=+0.378789633,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-145.ec2.internal,}"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.951061    2570 policy_none.go:47] "Start"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.962054    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-145.ec2.internal.18c0072d9e3e0e82  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-145.ec2.internal,UID:ip-10-0-130-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-145.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-145.ec2.internal,},FirstTimestamp:2026-07-07 14:09:04.948506242 +0000 UTC m=+0.378804594,LastTimestamp:2026-07-07 14:09:04.948506242 +0000 UTC m=+0.378804594,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-145.ec2.internal,}"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.975713    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-145.ec2.internal.18c0072d9e3e586b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-145.ec2.internal,UID:ip-10-0-130-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-145.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-145.ec2.internal,},FirstTimestamp:2026-07-07 14:09:04.948525163 +0000 UTC m=+0.378823515,LastTimestamp:2026-07-07 14:09:04.948525163 +0000 UTC m=+0.378823515,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-145.ec2.internal,}"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.976497    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lnptr"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.983322    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lnptr"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.987998    2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-145.ec2.internal.18c0072d9e3dd40e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-145.ec2.internal.18c0072d9e3dd40e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-145.ec2.internal,UID:ip-10-0-130-145.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-145.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-145.ec2.internal,},FirstTimestamp:2026-07-07 14:09:04.948491278 +0000 UTC m=+0.378789633,LastTimestamp:2026-07-07 14:09:04.949857208 +0000 UTC m=+0.380155563,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-145.ec2.internal,}"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994234    2570 manager.go:337] "Starting Device Plugin manager"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.994257    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994265    2570 server.go:87] "Starting device plugin registration server"
Jul 07 14:09:05.256269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994430    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994441    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994522    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994570    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:04.994575    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.994974    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:04.995013    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.093070    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.094171    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.094199    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.094216    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:05.094242    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.094482    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.095218    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.095235    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.095246    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.095272    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.097355    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.110790    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:05.110811    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-145.ec2.internal\": node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:05.193169    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.195202    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"]
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.195248    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.197696    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.197726    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.197735    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.199888    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.199998    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200019    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200488    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200514    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200524    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200491    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200581    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.200590    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.256781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.202622    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.202640    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.203228    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.203269    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.203283    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:05.227228    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-145.ec2.internal\" not found" node="ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:05.231375    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-145.ec2.internal\" not found" node="ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.232379    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/635bab47631366238f21adec9b508f0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal\" (UID: \"635bab47631366238f21adec9b508f0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.232400    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/635bab47631366238f21adec9b508f0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal\" (UID: \"635bab47631366238f21adec9b508f0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:05.257701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.232416    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68eba935587531e5b65d894a8d258574-config\") pod \"kube-apiserver-proxy-ip-10-0-130-145.ec2.internal\" (UID: \"68eba935587531e5b65d894a8d258574\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:05.293638    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.332632    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/635bab47631366238f21adec9b508f0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal\" (UID: \"635bab47631366238f21adec9b508f0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.332654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/635bab47631366238f21adec9b508f0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal\" (UID: \"635bab47631366238f21adec9b508f0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.332669    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68eba935587531e5b65d894a8d258574-config\") pod \"kube-apiserver-proxy-ip-10-0-130-145.ec2.internal\" (UID: \"68eba935587531e5b65d894a8d258574\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.332692    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68eba935587531e5b65d894a8d258574-config\") pod \"kube-apiserver-proxy-ip-10-0-130-145.ec2.internal\" (UID: \"68eba935587531e5b65d894a8d258574\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.332726    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/635bab47631366238f21adec9b508f0a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal\" (UID: \"635bab47631366238f21adec9b508f0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:05.332734    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/635bab47631366238f21adec9b508f0a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal\" (UID: \"635bab47631366238f21adec9b508f0a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.558061 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:06.535699    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:06.732867 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:06.636236    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:06.732867 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:06.671078    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.732867 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:06.675621    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:06.958584 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:06.736995    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:06.958584 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:06.837415    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:06.958584 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:06.937820    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-145.ec2.internal\" not found"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:06.981289    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.010252    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.010344    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.010376    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.057196    2570 apiserver.go:52] "Watching apiserver"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.073449    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.073455    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.094047    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.094421    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-cn8n9","kube-system/konnectivity-agent-8hx6v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc","openshift-dns/node-resolver-6nzt4","openshift-image-registry/node-ca-47k5s","openshift-multus/multus-additional-cni-plugins-jdw9x","openshift-multus/network-metrics-daemon-x8nq2","openshift-network-operator/iptables-alerter-r8m8z","openshift-cluster-node-tuning-operator/tuned-rf5kb","openshift-multus/multus-2zjck","openshift-network-diagnostics/network-check-target-v6sph","openshift-ovn-kubernetes/ovnkube-node-mns74"]
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.096727    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.096788    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.098683    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.101165    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.103604    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.103726    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.103824    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.106125    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.107424    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.109545    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.109590    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.111697    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.113873    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.116025    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.116959    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jsxvv\""
Jul 07 14:09:07.135705 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.118198    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.118261    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.118747    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.118920    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119064    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119245    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119416    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119442    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119574    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119585    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119772    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119891    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zh9wx\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.119987    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.124048    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.124088    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.125930    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:04:06 +0000 UTC" deadline="2028-01-22 06:59:38.651986973 +0000 UTC"
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.125946    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13528h50m31.526043304s"
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.126206    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal"]
Jul 07 14:09:07.137048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.126297    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.144551    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j27zg\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.145102    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.145223    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xpt77\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.145225    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.145254    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l5r75\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.145605    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-25dtn\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.149956    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.149957    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.153883    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5srvs\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.153932    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154066    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154122    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154258    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154316    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154334    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6t9jr\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154333    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154378    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ft4c9\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154413    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154430    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154436    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154444    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.154487    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.162440    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.172777    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.175313    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal"]
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.176182    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179744    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-systemd\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.294780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179765    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnn9c\" (UniqueName: \"kubernetes.io/projected/d7f78da8-c8e1-47ff-819b-313e2151d952-kube-api-access-cnn9c\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gs4l\" (UniqueName: \"kubernetes.io/projected/32823771-b3e3-4806-a6eb-4e66616a5b48-kube-api-access-4gs4l\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179794    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-system-cni-dir\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179809    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-cni-binary-copy\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179823    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrtzl\" (UniqueName: \"kubernetes.io/projected/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-kube-api-access-wrtzl\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179849    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-etc-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179869    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b887d71-ef48-4b43-b8d2-700d180d40ff-dbus\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179890    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32823771-b3e3-4806-a6eb-4e66616a5b48-hosts-file\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179904    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179920    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-kubernetes\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179935    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-daemon-config\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179950    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-var-lib-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179968    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-ovnkube-config\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.179983    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.295961 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180007    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-host\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180028    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-k8s-cni-cncf-io\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180043    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180057    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-netns\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180072    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180087    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-etc-selinux\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64016df9-5267-45ef-bd4f-2b13535b77c4-host\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180113    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180127    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-run\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-etc-kubernetes\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180180    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-env-overrides\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180195    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsdzp\" (UniqueName: \"kubernetes.io/projected/64016df9-5267-45ef-bd4f-2b13535b77c4-kube-api-access-vsdzp\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-os-release\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180230    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-kubelet\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.296967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180252    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-device-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180273    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-systemd-units\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180287    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5979d983-b592-460c-a540-bf0049e7071e-ovn-node-metrics-cert\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180301    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-socket-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180316    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64016df9-5267-45ef-bd4f-2b13535b77c4-serviceca\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180336    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-lib-modules\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180362    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-os-release\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180379    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b887d71-ef48-4b43-b8d2-700d180d40ff-kubelet-config\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180393    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-systemd\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180412    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-kubelet\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180425    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-node-log\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180444    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd4hm\" (UniqueName: \"kubernetes.io/projected/f2436106-b7a9-47b1-94ba-0b687b02fa92-kube-api-access-pd4hm\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32823771-b3e3-4806-a6eb-4e66616a5b48-tmp-dir\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180474    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-modprobe-d\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.297605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180496    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-var-lib-kubelet\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180520    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-log-socket\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180545    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-cni-netd\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180572    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-iptables-alerter-script\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180612    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysctl-d\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180625    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-cni-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180650    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-run-ovn-kubernetes\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180663    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-cni-bin\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180676    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-cnibin\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180702    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180719    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr2qx\" (UniqueName: \"kubernetes.io/projected/015c7f85-80c7-42cf-8c77-6f22642344c1-kube-api-access-lr2qx\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180739    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6gcb\" (UniqueName: \"kubernetes.io/projected/f3888bea-77ca-4f33-9126-f1673c37beb9-kube-api-access-h6gcb\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.298276 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180753    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-host-slash\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180771    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysconfig\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180792    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-socket-dir-parent\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180814    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c-agent-certs\") pod \"konnectivity-agent-8hx6v\" (UID: \"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c\") " pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180831    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxpcl\" (UniqueName: \"kubernetes.io/projected/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-kube-api-access-gxpcl\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180857    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysctl-conf\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180880    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-cnibin\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180898    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-hostroot\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180918    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-multus-certs\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180943    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-ovn\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180958    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180973    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.180985    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-tuned\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181002    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-slash\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.298805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181022    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181035    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-system-cni-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181047    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-cni-bin\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181059    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-cni-multus\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181075    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-run-netns\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181112    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-ovnkube-script-lib\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181125    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ns9j\" (UniqueName: \"kubernetes.io/projected/5979d983-b592-460c-a540-bf0049e7071e-kube-api-access-5ns9j\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181139    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c-konnectivity-ca\") pod \"konnectivity-agent-8hx6v\" (UID: \"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c\") " pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181170    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-registration-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181188    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-sys-fs\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-sys\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181222    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2436106-b7a9-47b1-94ba-0b687b02fa92-tmp\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.181242    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-conf-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.200213    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hnstm"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.210859    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hnstm"
Jul 07 14:09:07.299336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-netns\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281555    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281569    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-etc-selinux\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281583    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64016df9-5267-45ef-bd4f-2b13535b77c4-host\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281602    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281637    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-netns\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281637    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281662    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-run\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281669    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64016df9-5267-45ef-bd4f-2b13535b77c4-host\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281692    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-etc-selinux\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281692    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-etc-kubernetes\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281729    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-etc-kubernetes\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281757    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-run\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281732    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-env-overrides\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281789    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsdzp\" (UniqueName: \"kubernetes.io/projected/64016df9-5267-45ef-bd4f-2b13535b77c4-kube-api-access-vsdzp\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.299870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281805    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-os-release\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281818    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-kubelet\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281838    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-device-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281868    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-systemd-units\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281875    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-os-release\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5979d983-b592-460c-a540-bf0049e7071e-ovn-node-metrics-cert\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281890    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-kubelet\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281898    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-socket-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281912    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64016df9-5267-45ef-bd4f-2b13535b77c4-serviceca\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281923    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-device-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281987    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-lib-modules\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.281925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-lib-modules\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282063    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-os-release\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282081    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-socket-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.300397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b887d71-ef48-4b43-b8d2-700d180d40ff-kubelet-config\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282121    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-systemd\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282121    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282128    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7b887d71-ef48-4b43-b8d2-700d180d40ff-kubelet-config\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-kubelet\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282177    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-env-overrides\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282181    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-node-log\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282196    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-os-release\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd4hm\" (UniqueName: \"kubernetes.io/projected/f2436106-b7a9-47b1-94ba-0b687b02fa92-kube-api-access-pd4hm\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282227    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-node-log\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282223    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282246    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32823771-b3e3-4806-a6eb-4e66616a5b48-tmp-dir\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282248    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-systemd-units\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282242    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64016df9-5267-45ef-bd4f-2b13535b77c4-serviceca\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282222    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-systemd\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.300906 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282271    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-modprobe-d\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282293    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-var-lib-kubelet\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282322    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-log-socket\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282329    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-modprobe-d\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282339    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-cni-netd\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282246    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-kubelet\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282363    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282397    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-iptables-alerter-script\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282406    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-var-lib-kubelet\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysctl-d\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282370    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-log-socket\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282445    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-cni-netd\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282453    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-cni-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282468    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32823771-b3e3-4806-a6eb-4e66616a5b48-tmp-dir\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282477    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-run-ovn-kubernetes\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.301875 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282505    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-cni-bin\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282514    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-run-ovn-kubernetes\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282524    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysctl-d\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.282483    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282526    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-cnibin\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282572    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-cni-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282594    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282562    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-cnibin\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282598    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-cni-bin\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.282646    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:09:07.782605507 +0000 UTC m=+2.071514603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.282666    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.282720    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:07.782704348 +0000 UTC m=+2.071613433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282736    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr2qx\" (UniqueName: \"kubernetes.io/projected/015c7f85-80c7-42cf-8c77-6f22642344c1-kube-api-access-lr2qx\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6gcb\" (UniqueName: \"kubernetes.io/projected/f3888bea-77ca-4f33-9126-f1673c37beb9-kube-api-access-h6gcb\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.302800 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282780    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-host-slash\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282788    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-iptables-alerter-script\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282804    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysconfig\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282836    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-host-slash\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282839    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-socket-dir-parent\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282862    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c-agent-certs\") pod \"konnectivity-agent-8hx6v\" (UID: \"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c\") " pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282877    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysconfig\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxpcl\" (UniqueName: \"kubernetes.io/projected/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-kube-api-access-gxpcl\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282904    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysctl-conf\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282926    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-cnibin\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282945    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-hostroot\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-multus-certs\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.282995    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-ovn\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283018    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.303682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283026    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-socket-dir-parent\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283043    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283059    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-hostroot\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283064    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-tuned\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283101    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-slash\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283113    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283127    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283168    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-cnibin\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283176    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-multus-certs\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283181    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-system-cni-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283219    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-ovn\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283234    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-slash\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-cni-bin\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283795    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-cni-bin\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283955    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.304554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.283137    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-sysctl-conf\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284206    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-cni-multus\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284291    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-run-netns\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284340    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-var-lib-cni-multus\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284564    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-host-run-netns\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284600    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-ovnkube-script-lib\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284641    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ns9j\" (UniqueName: \"kubernetes.io/projected/5979d983-b592-460c-a540-bf0049e7071e-kube-api-access-5ns9j\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284675    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c-konnectivity-ca\") pod \"konnectivity-agent-8hx6v\" (UID: \"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c\") " pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284707    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-registration-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-sys-fs\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284763    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-sys\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284795    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2436106-b7a9-47b1-94ba-0b687b02fa92-tmp\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284823    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-conf-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-systemd\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.305524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnn9c\" (UniqueName: \"kubernetes.io/projected/d7f78da8-c8e1-47ff-819b-313e2151d952-kube-api-access-cnn9c\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284913    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gs4l\" (UniqueName: \"kubernetes.io/projected/32823771-b3e3-4806-a6eb-4e66616a5b48-kube-api-access-4gs4l\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.284949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-system-cni-dir\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285181    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-cni-binary-copy\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285213    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrtzl\" (UniqueName: \"kubernetes.io/projected/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-kube-api-access-wrtzl\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-etc-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285273    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b887d71-ef48-4b43-b8d2-700d180d40ff-dbus\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285400    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32823771-b3e3-4806-a6eb-4e66616a5b48-hosts-file\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285432    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285462    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-kubernetes\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285487    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-daemon-config\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285516    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-var-lib-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285539    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-registration-dir\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285551    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-ovnkube-config\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.306472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7f78da8-c8e1-47ff-819b-313e2151d952-sys-fs\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285700    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-host\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285773    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-k8s-cni-cncf-io\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285834    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5979d983-b592-460c-a540-bf0049e7071e-ovn-node-metrics-cert\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285942    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-etc-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285989    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-host-run-k8s-cni-cncf-io\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286022    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286160    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286211    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-run-systemd\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286286    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-system-cni-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286394    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-ovnkube-config\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286479    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-sys\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286687    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c-agent-certs\") pod \"konnectivity-agent-8hx6v\" (UID: \"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c\") " pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286706    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5979d983-b592-460c-a540-bf0049e7071e-ovnkube-script-lib\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.307330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286830    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5979d983-b592-460c-a540-bf0049e7071e-var-lib-openvswitch\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.285522    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-conf-dir\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286872    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-host\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.286947    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287058    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/32823771-b3e3-4806-a6eb-4e66616a5b48-hosts-file\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287066    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-kubernetes\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287353    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2436106-b7a9-47b1-94ba-0b687b02fa92-etc-tuned\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287510    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7b887d71-ef48-4b43-b8d2-700d180d40ff-dbus\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287589    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f3888bea-77ca-4f33-9126-f1673c37beb9-system-cni-dir\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287883    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-cni-binary-copy\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.287990    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c-konnectivity-ca\") pod \"konnectivity-agent-8hx6v\" (UID: \"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c\") " pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.288047    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2436106-b7a9-47b1-94ba-0b687b02fa92-tmp\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.288062    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f3888bea-77ca-4f33-9126-f1673c37beb9-cni-binary-copy\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.289421    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-multus-daemon-config\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.302129    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsdzp\" (UniqueName: \"kubernetes.io/projected/64016df9-5267-45ef-bd4f-2b13535b77c4-kube-api-access-vsdzp\") pod \"node-ca-47k5s\" (UID: \"64016df9-5267-45ef-bd4f-2b13535b77c4\") " pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.308165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.302710    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr2qx\" (UniqueName: \"kubernetes.io/projected/015c7f85-80c7-42cf-8c77-6f22642344c1-kube-api-access-lr2qx\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.303258    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.303278    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.303291    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.303344    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:09:07.803329497 +0000 UTC m=+2.092238594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.304281    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxpcl\" (UniqueName: \"kubernetes.io/projected/bd68a3bd-c3d7-4694-bc4d-70e61be7d632-kube-api-access-gxpcl\") pod \"iptables-alerter-r8m8z\" (UID: \"bd68a3bd-c3d7-4694-bc4d-70e61be7d632\") " pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.305612    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6gcb\" (UniqueName: \"kubernetes.io/projected/f3888bea-77ca-4f33-9126-f1673c37beb9-kube-api-access-h6gcb\") pod \"multus-additional-cni-plugins-jdw9x\" (UID: \"f3888bea-77ca-4f33-9126-f1673c37beb9\") " pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.305987    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gs4l\" (UniqueName: \"kubernetes.io/projected/32823771-b3e3-4806-a6eb-4e66616a5b48-kube-api-access-4gs4l\") pod \"node-resolver-6nzt4\" (UID: \"32823771-b3e3-4806-a6eb-4e66616a5b48\") " pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.306242    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrtzl\" (UniqueName: \"kubernetes.io/projected/be3f27ea-f77a-4be0-b234-5607a8b9ffcc-kube-api-access-wrtzl\") pod \"multus-2zjck\" (UID: \"be3f27ea-f77a-4be0-b234-5607a8b9ffcc\") " pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.306568    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ns9j\" (UniqueName: \"kubernetes.io/projected/5979d983-b592-460c-a540-bf0049e7071e-kube-api-access-5ns9j\") pod \"ovnkube-node-mns74\" (UID: \"5979d983-b592-460c-a540-bf0049e7071e\") " pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.306575    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnn9c\" (UniqueName: \"kubernetes.io/projected/d7f78da8-c8e1-47ff-819b-313e2151d952-kube-api-access-cnn9c\") pod \"aws-ebs-csi-driver-node-gv9pc\" (UID: \"d7f78da8-c8e1-47ff-819b-313e2151d952\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.309031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.306811    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd4hm\" (UniqueName: \"kubernetes.io/projected/f2436106-b7a9-47b1-94ba-0b687b02fa92-kube-api-access-pd4hm\") pod \"tuned-rf5kb\" (UID: \"f2436106-b7a9-47b1-94ba-0b687b02fa92\") " pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.440327 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.411693    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:09:07.449839 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.446230    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.451428    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc"
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.461494    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f78da8_c8e1_47ff_819b_313e2151d952.slice/crio-5f3783f5032da92fdf8d3e6cf7b59d1f5ac56b8e979af058fc56f213627ab102 WatchSource:0}: Error finding container 5f3783f5032da92fdf8d3e6cf7b59d1f5ac56b8e979af058fc56f213627ab102: Status 404 returned error can't find the container with id 5f3783f5032da92fdf8d3e6cf7b59d1f5ac56b8e979af058fc56f213627ab102
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.461754    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc24eb5_cdcf_429c_81b4_1ff5ec2edf9c.slice/crio-411e877ba7fe35f979627165f128b080b3455544d020c3b40356b4a1abbdf407 WatchSource:0}: Error finding container 411e877ba7fe35f979627165f128b080b3455544d020c3b40356b4a1abbdf407: Status 404 returned error can't find the container with id 411e877ba7fe35f979627165f128b080b3455544d020c3b40356b4a1abbdf407
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.462546    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635bab47631366238f21adec9b508f0a.slice/crio-96ae38d81db31a55245650666176f9aaa0fc834e900117d886f8376592d19b46 WatchSource:0}: Error finding container 96ae38d81db31a55245650666176f9aaa0fc834e900117d886f8376592d19b46: Status 404 returned error can't find the container with id 96ae38d81db31a55245650666176f9aaa0fc834e900117d886f8376592d19b46
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.468189    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.477813    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6nzt4"
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.485967    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32823771_b3e3_4806_a6eb_4e66616a5b48.slice/crio-d61536d7a9b9e4097c5d6eded4c705a4d2d861b06ead82c5e2c1986767b6e992 WatchSource:0}: Error finding container d61536d7a9b9e4097c5d6eded4c705a4d2d861b06ead82c5e2c1986767b6e992: Status 404 returned error can't find the container with id d61536d7a9b9e4097c5d6eded4c705a4d2d861b06ead82c5e2c1986767b6e992
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.491807    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-47k5s"
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.498611    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64016df9_5267_45ef_bd4f_2b13535b77c4.slice/crio-3de7ef59a94900b1ba781dddb604b7edcfeb255abb01a273c189980fab2b0562 WatchSource:0}: Error finding container 3de7ef59a94900b1ba781dddb604b7edcfeb255abb01a273c189980fab2b0562: Status 404 returned error can't find the container with id 3de7ef59a94900b1ba781dddb604b7edcfeb255abb01a273c189980fab2b0562
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.510971    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jdw9x"
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.514678    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r8m8z"
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.518332    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3888bea_77ca_4f33_9126_f1673c37beb9.slice/crio-d8ebb63561948fb548588e1d16620215faa5eaf9b19dd83c2e8746b66eaf9c1a WatchSource:0}: Error finding container d8ebb63561948fb548588e1d16620215faa5eaf9b19dd83c2e8746b66eaf9c1a: Status 404 returned error can't find the container with id d8ebb63561948fb548588e1d16620215faa5eaf9b19dd83c2e8746b66eaf9c1a
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.522040    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd68a3bd_c3d7_4694_bc4d_70e61be7d632.slice/crio-44b252aaf9ee613123f9464bb98c4f26759c98bcf96ff46c5b7625c1c66059b4 WatchSource:0}: Error finding container 44b252aaf9ee613123f9464bb98c4f26759c98bcf96ff46c5b7625c1c66059b4: Status 404 returned error can't find the container with id 44b252aaf9ee613123f9464bb98c4f26759c98bcf96ff46c5b7625c1c66059b4
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.550489    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68eba935587531e5b65d894a8d258574.slice/crio-9559bd9111a715af138e3ab39fd4851cfa0453d47d5cd0d7fa4ea2f765f89207 WatchSource:0}: Error finding container 9559bd9111a715af138e3ab39fd4851cfa0453d47d5cd0d7fa4ea2f765f89207: Status 404 returned error can't find the container with id 9559bd9111a715af138e3ab39fd4851cfa0453d47d5cd0d7fa4ea2f765f89207
Jul 07 14:09:07.613818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.555417    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rf5kb"
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.562351    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2436106_b7a9_47b1_94ba_0b687b02fa92.slice/crio-602603792d2e5f1ede3d89914380cbcdd0964b927ba96a1b328fdd06cece746a WatchSource:0}: Error finding container 602603792d2e5f1ede3d89914380cbcdd0964b927ba96a1b328fdd06cece746a: Status 404 returned error can't find the container with id 602603792d2e5f1ede3d89914380cbcdd0964b927ba96a1b328fdd06cece746a
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.575282    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2zjck"
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.579759    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.582300    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3f27ea_f77a_4be0_b234_5607a8b9ffcc.slice/crio-77bb4249b80e46eee62e6e8255670b505b0ffaa6aca716183352e9b617d8eccb WatchSource:0}: Error finding container 77bb4249b80e46eee62e6e8255670b505b0ffaa6aca716183352e9b617d8eccb: Status 404 returned error can't find the container with id 77bb4249b80e46eee62e6e8255670b505b0ffaa6aca716183352e9b617d8eccb
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:07.587059    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5979d983_b592_460c_a540_bf0049e7071e.slice/crio-086098e69d7bbb933f63858ac3d1c819d2b3a73ae7311b9ad92af56b771ba172 WatchSource:0}: Error finding container 086098e69d7bbb933f63858ac3d1c819d2b3a73ae7311b9ad92af56b771ba172: Status 404 returned error can't find the container with id 086098e69d7bbb933f63858ac3d1c819d2b3a73ae7311b9ad92af56b771ba172
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.588744    2570 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-mns74_openshift-ovn-kubernetes(5979d983-b592-460c-a540-bf0049e7071e): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.590445    2570 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-mns74_openshift-ovn-kubernetes(5979d983-b592-460c-a540-bf0049e7071e): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.594652    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" podUID="5979d983-b592-460c-a540-bf0049e7071e"
Jul 07 14:09:07.617639 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.611127    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:09:07.840125 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.788266    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:07.840125 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.788316    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:07.840125 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.788391    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:07.840125 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.788402    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:07.840125 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.788434    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:08.788420597 +0000 UTC m=+3.077329678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:07.840125 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.788447    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:09:08.788440859 +0000 UTC m=+3.077349935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:08.031508 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:07.889263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:08.031508 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.889389    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:08.031508 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.889403    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:08.031508 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.889411    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:08.031508 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:07.889446    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:09:08.889434534 +0000 UTC m=+3.178343624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.213715    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:04:07 +0000 UTC" deadline="2028-03-24 04:01:42.29004291 +0000 UTC"
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.213763    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15013h52m34.076283656s"
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.242022    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hx6v" event={"ID":"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c","Type":"ContainerStarted","Data":"411e877ba7fe35f979627165f128b080b3455544d020c3b40356b4a1abbdf407"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.244861    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal" event={"ID":"635bab47631366238f21adec9b508f0a","Type":"ContainerStarted","Data":"96ae38d81db31a55245650666176f9aaa0fc834e900117d886f8376592d19b46"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.247419    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r8m8z" event={"ID":"bd68a3bd-c3d7-4694-bc4d-70e61be7d632","Type":"ContainerStarted","Data":"44b252aaf9ee613123f9464bb98c4f26759c98bcf96ff46c5b7625c1c66059b4"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.249630    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerStarted","Data":"d8ebb63561948fb548588e1d16620215faa5eaf9b19dd83c2e8746b66eaf9c1a"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.250859    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6nzt4" event={"ID":"32823771-b3e3-4806-a6eb-4e66616a5b48","Type":"ContainerStarted","Data":"d61536d7a9b9e4097c5d6eded4c705a4d2d861b06ead82c5e2c1986767b6e992"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.252232    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc" event={"ID":"d7f78da8-c8e1-47ff-819b-313e2151d952","Type":"ContainerStarted","Data":"5f3783f5032da92fdf8d3e6cf7b59d1f5ac56b8e979af058fc56f213627ab102"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.253554    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"086098e69d7bbb933f63858ac3d1c819d2b3a73ae7311b9ad92af56b771ba172"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.254900    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2zjck" event={"ID":"be3f27ea-f77a-4be0-b234-5607a8b9ffcc","Type":"ContainerStarted","Data":"77bb4249b80e46eee62e6e8255670b505b0ffaa6aca716183352e9b617d8eccb"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.256400    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rf5kb" event={"ID":"f2436106-b7a9-47b1-94ba-0b687b02fa92","Type":"ContainerStarted","Data":"602603792d2e5f1ede3d89914380cbcdd0964b927ba96a1b328fdd06cece746a"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.257898    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal" event={"ID":"68eba935587531e5b65d894a8d258574","Type":"ContainerStarted","Data":"9559bd9111a715af138e3ab39fd4851cfa0453d47d5cd0d7fa4ea2f765f89207"}
Jul 07 14:09:08.260558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.259228    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-47k5s" event={"ID":"64016df9-5267-45ef-bd4f-2b13535b77c4","Type":"ContainerStarted","Data":"3de7ef59a94900b1ba781dddb604b7edcfeb255abb01a273c189980fab2b0562"}
Jul 07 14:09:08.278427 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.278095    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" podUID="5979d983-b592-460c-a540-bf0049e7071e"
Jul 07 14:09:08.349619 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.349597    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:09:08.795124 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.795085    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:08.795312 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.795137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:08.795312 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.795304    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:08.795426 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.795359    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:10.795340072 +0000 UTC m=+5.084249155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:08.795828 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.795708    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:08.795828 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.795752    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:09:10.79573897 +0000 UTC m=+5.084648049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:08.896040 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:08.896009    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:08.896254 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.896168    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:08.896254 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.896191    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:08.896254 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.896204    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:08.896441 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:08.896261    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:09:10.896246084 +0000 UTC m=+5.185155174 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:09.214567 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:09.214435    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:04:07 +0000 UTC" deadline="2028-02-15 08:09:46.033828165 +0000 UTC"
Jul 07 14:09:09.214567 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:09.214464    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14106h0m36.81936867s"
Jul 07 14:09:09.236350 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:09.236324    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:09.236497 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:09.236436    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:09.236822 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:09.236795    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:09.236915 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:09.236884    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:09.236977 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:09.236945    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:09.237039 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:09.237023    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:10.143491 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:10.143466    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:09:10.813042 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:10.812377    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:10.813042 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:10.812434    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:10.813042 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.812608    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:10.813042 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.812665    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:14.812646988 +0000 UTC m=+9.101556067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:10.813042 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.812807    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:10.813042 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.812863    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:09:14.812845806 +0000 UTC m=+9.101754896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:10.913560 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:10.913446    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:10.913706 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.913598    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:10.913706 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.913617    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:10.913706 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.913630    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:10.913706 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:10.913690    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:09:14.913673267 +0000 UTC m=+9.202582360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:11.236041 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:11.235963    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:11.236474 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:11.236085    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:11.236536 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:11.236478    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:11.236652 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:11.236609    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:11.236717 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:11.236684    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:11.236817 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:11.236757    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:13.235798 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:13.235769    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:13.235798 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:13.235796    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:13.236294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:13.235769    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:13.236294 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:13.235893    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:13.236294 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:13.235949    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:13.236294 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:13.236024    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:14.841686 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:14.841572    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:14.842088 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.841702    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:14.842088 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.841766    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:09:22.84174641 +0000 UTC m=+17.130655491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:14.842088 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:14.841792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:14.842088 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.841885    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:14.842088 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.841919    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:22.841907572 +0000 UTC m=+17.130816650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:14.942557 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:14.942476    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:14.942722 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.942641    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:14.942722 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.942665    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:14.942722 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.942674    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:14.942883 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:14.942729    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:09:22.942709599 +0000 UTC m=+17.231618675 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:15.236603 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:15.236527    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:15.236603 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:15.236569    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:15.236603 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:15.236569    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:15.236871 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:15.236636    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:15.236871 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:15.236749    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:15.236871 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:15.236848    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:17.236108 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.235872    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:17.236528 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.235874    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:17.236528 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:17.236179    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:17.236528 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:17.236280    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:17.236528 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.235986    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:17.236528 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:17.236407    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:17.280138 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.280109    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc" event={"ID":"d7f78da8-c8e1-47ff-819b-313e2151d952","Type":"ContainerStarted","Data":"038d934f7dce618ef0c46f7009079f32d7fbaeea8bb93030d0be34624dc688d6"}
Jul 07 14:09:17.281914 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.281869    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rf5kb" event={"ID":"f2436106-b7a9-47b1-94ba-0b687b02fa92","Type":"ContainerStarted","Data":"a1ecb325f294fac71cd03d3896e07edc5773fcbbd1a8cc9c25e84ea379c07594"}
Jul 07 14:09:17.283789 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.283758    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal" event={"ID":"68eba935587531e5b65d894a8d258574","Type":"ContainerStarted","Data":"d00025db210d7f82994dcfff1843fb3f184b1ae4c3ba82612f9fffa38b8fcfc1"}
Jul 07 14:09:17.286025 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.285995    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-47k5s" event={"ID":"64016df9-5267-45ef-bd4f-2b13535b77c4","Type":"ContainerStarted","Data":"29986bf53290714679c7699823f9a0c9c1c653de70023c873687909f2dd2f7ad"}
Jul 07 14:09:17.288161 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.288116    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8hx6v" event={"ID":"ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c","Type":"ContainerStarted","Data":"b92d6acbcbef7664f934fbbefb6a94d383942b38664604a8c2b5585c734a2eef"}
Jul 07 14:09:17.289622 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.289600    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal" event={"ID":"635bab47631366238f21adec9b508f0a","Type":"ContainerStarted","Data":"a6062e9dde43ecbbb5862be8b436c195bd5227019fd3b803e4e9af0d5e73eeb5"}
Jul 07 14:09:17.291121 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.291090    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerStarted","Data":"ea393c03dc424da241bd7d1ac5773170e775616821820afdd12dac5685b0f21c"}
Jul 07 14:09:17.292904 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.292884    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6nzt4" event={"ID":"32823771-b3e3-4806-a6eb-4e66616a5b48","Type":"ContainerStarted","Data":"45b71e96f3094cd33dfa5aedc8092bc8aa716e87db2b9b3408b673150b60bbef"}
Jul 07 14:09:17.328900 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.328864    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rf5kb" podStartSLOduration=2.285162774 podStartE2EDuration="11.328851618s" totalImagesPullingTime="9.043688844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:17.306143241 +0000 UTC m=+11.595052360" watchObservedRunningTime="2026-07-07 14:09:17.328851618 +0000 UTC m=+11.617760715"
Jul 07 14:09:17.376157 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.376098    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-145.ec2.internal" podStartSLOduration=10.376085456 podStartE2EDuration="10.376085456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:17.355165433 +0000 UTC m=+11.644074522" watchObservedRunningTime="2026-07-07 14:09:17.376085456 +0000 UTC m=+11.664994585"
Jul 07 14:09:17.376272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.376216    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8hx6v" podStartSLOduration=2.260336644 podStartE2EDuration="11.376205221s" totalImagesPullingTime="9.115868577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:17.374059315 +0000 UTC m=+11.662968414" watchObservedRunningTime="2026-07-07 14:09:17.376205221 +0000 UTC m=+11.665114320"
Jul 07 14:09:17.412016 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.411843    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6nzt4" podStartSLOduration=2.317733573 podStartE2EDuration="11.411829705s" totalImagesPullingTime="9.094096132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:17.3925461 +0000 UTC m=+11.681455198" watchObservedRunningTime="2026-07-07 14:09:17.411829705 +0000 UTC m=+11.700738803"
Jul 07 14:09:17.412110 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:17.412053    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-47k5s" podStartSLOduration=2.34736551 podStartE2EDuration="11.412045778s" totalImagesPullingTime="9.064680268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:17.411082465 +0000 UTC m=+11.699991565" watchObservedRunningTime="2026-07-07 14:09:17.412045778 +0000 UTC m=+11.700954878"
Jul 07 14:09:18.296144 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:18.296058    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r8m8z" event={"ID":"bd68a3bd-c3d7-4694-bc4d-70e61be7d632","Type":"ContainerStarted","Data":"fa732a2a7213adceb9f29ef88e169ad669e9a3797f3ee9496bf26141d7def075"}
Jul 07 14:09:18.312163 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:18.312097    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r8m8z" podStartSLOduration=3.2526112449999998 podStartE2EDuration="12.312077902s" totalImagesPullingTime="9.059466657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:18.311392814 +0000 UTC m=+12.600301914" watchObservedRunningTime="2026-07-07 14:09:18.312077902 +0000 UTC m=+12.600987001"
Jul 07 14:09:18.594125 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:18.594060    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:18.594790 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:18.594772    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:19.236718 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:19.236683    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:19.236891 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:19.236731    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:19.236891 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:19.236700    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:19.236891 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:19.236803    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:19.237058 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:19.236890    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:19.237058 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:19.237024    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:19.371030 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:19.371005    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:19.371581 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:19.371564    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8hx6v"
Jul 07 14:09:21.236056 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.236031    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:21.236779 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.236079    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:21.236779 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:21.236176    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:21.236779 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.236273    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:21.236779 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:21.236362    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:21.236779 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:21.236475    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:21.302334 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.302310    2570 generic.go:356] "Generic (PLEG): container finished" podID="635bab47631366238f21adec9b508f0a" containerID="a6062e9dde43ecbbb5862be8b436c195bd5227019fd3b803e4e9af0d5e73eeb5" exitCode=0
Jul 07 14:09:21.302447 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.302377    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal" event={"ID":"635bab47631366238f21adec9b508f0a","Type":"ContainerDied","Data":"a6062e9dde43ecbbb5862be8b436c195bd5227019fd3b803e4e9af0d5e73eeb5"}
Jul 07 14:09:21.303860 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.303842    2570 generic.go:356] "Generic (PLEG): container finished" podID="f3888bea-77ca-4f33-9126-f1673c37beb9" containerID="ea393c03dc424da241bd7d1ac5773170e775616821820afdd12dac5685b0f21c" exitCode=0
Jul 07 14:09:21.303944 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.303902    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerDied","Data":"ea393c03dc424da241bd7d1ac5773170e775616821820afdd12dac5685b0f21c"}
Jul 07 14:09:21.305312 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.305240    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2zjck" event={"ID":"be3f27ea-f77a-4be0-b234-5607a8b9ffcc","Type":"ContainerStarted","Data":"995523bcaeb2b34900674dbf5e04c86653afb632b93d28593ed7e0ee4e88007d"}
Jul 07 14:09:21.364245 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.364201    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2zjck" podStartSLOduration=2.072650869 podStartE2EDuration="15.364189261s" totalImagesPullingTime="13.291538392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:21.364021448 +0000 UTC m=+15.652930546" watchObservedRunningTime="2026-07-07 14:09:21.364189261 +0000 UTC m=+15.653098361"
Jul 07 14:09:21.572895 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:21.572857    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 14:09:22.143808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.143707    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:09:21.572876476Z","UUID":"67e3ac98-28b1-422d-8935-371381b0af4d","Handler":null,"Name":"","Endpoint":""}
Jul 07 14:09:22.145666 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.145646    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 14:09:22.145801 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.145685    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 14:09:22.310239 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.310206    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal" event={"ID":"635bab47631366238f21adec9b508f0a","Type":"ContainerStarted","Data":"6124cb11f5f202775417603d3b2609454361080017350f7be5e138c8486acfdc"}
Jul 07 14:09:22.315428 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.315402    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc" event={"ID":"d7f78da8-c8e1-47ff-819b-313e2151d952","Type":"ContainerStarted","Data":"4071318cd3746714e746a6aba38c1ee1ffe2a9824927db46d184f47873205767"}
Jul 07 14:09:22.328211 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.328141    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-145.ec2.internal" podStartSLOduration=15.328125337 podStartE2EDuration="15.328125337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:22.32721283 +0000 UTC m=+16.616121927" watchObservedRunningTime="2026-07-07 14:09:22.328125337 +0000 UTC m=+16.617034435"
Jul 07 14:09:22.902845 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.902809    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:22.903011 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:22.902853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:22.903011 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:22.902948    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:22.903011 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:22.902951    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:22.903011 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:22.903010    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:38.902991976 +0000 UTC m=+33.191901069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:22.903266 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:22.903027    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:09:38.903019389 +0000 UTC m=+33.191928469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:23.004074 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:23.004042    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:23.004258 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.004233    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:23.004258 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.004255    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:23.004413 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.004267    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:23.004413 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.004317    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:09:39.004300062 +0000 UTC m=+33.293209140 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:23.236375 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:23.236290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:23.236521 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:23.236405    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:23.236521 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.236415    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:23.236638 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.236528    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:23.236638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:23.236587    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:23.236735 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:23.236674    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:24.321765 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:24.321730    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc" event={"ID":"d7f78da8-c8e1-47ff-819b-313e2151d952","Type":"ContainerStarted","Data":"e89f5d186c4dc542547eb491226020ad29c51d394cccbaab0f84c16673e9d4c6"}
Jul 07 14:09:24.341511 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:24.341460    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gv9pc" podStartSLOduration=2.8574372820000002 podStartE2EDuration="18.341447303s" totalImagesPullingTime="15.484010021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:24.341420838 +0000 UTC m=+18.630329936" watchObservedRunningTime="2026-07-07 14:09:24.341447303 +0000 UTC m=+18.630356400"
Jul 07 14:09:25.236116 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:25.236090    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:25.236320 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:25.236089    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:25.236320 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:25.236211    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:25.236320 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:25.236090    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:25.236320 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:25.236267    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:25.236320 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:25.236317    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:27.236832 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:27.236799    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:27.237317 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:27.236916    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:27.237317 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:27.237262    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:27.237317 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:27.237282    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:27.237470 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:27.237348    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:27.237470 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:27.237398    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:29.236571 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:29.236544    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:29.237267 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:29.236658    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:29.237267 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:29.236654    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:29.237267 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:29.236694    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:29.237267 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:29.236776    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:29.237267 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:29.236858    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:31.236474 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:31.236442    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:31.236894 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:31.236496    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:31.236894 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:31.236495    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:31.236894 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:31.236590    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:31.236894 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:31.236664    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:31.236894 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:31.236708    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:33.236011 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:33.235980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:33.236470 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:33.236066    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:33.236470 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:33.236070    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:33.236470 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:33.236078    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:33.236470 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:33.236159    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:33.236470 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:33.236225    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:34.341863 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.341832    2570 generic.go:356] "Generic (PLEG): container finished" podID="f3888bea-77ca-4f33-9126-f1673c37beb9" containerID="89766e567c40151165f7dfb76ff485f1b62d7501ce25887205e4cc2971188d77" exitCode=0
Jul 07 14:09:34.342495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.341909    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerDied","Data":"89766e567c40151165f7dfb76ff485f1b62d7501ce25887205e4cc2971188d77"}
Jul 07 14:09:34.345234 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345214    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:09:34.345668 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345646    2570 generic.go:356] "Generic (PLEG): container finished" podID="5979d983-b592-460c-a540-bf0049e7071e" containerID="1a068dae809b3c713d45525b595e330573543556244d83f1699ce65975268866" exitCode=1
Jul 07 14:09:34.345748 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345680    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"87c55ba201ad13a8fd1b6e23090aa4f1e7841e59ab1924d0df020e99d1cf478b"}
Jul 07 14:09:34.345748 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345735    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"18ec30dda3e6308cfe400ff795e88d79da2ad65ba02ac758618e76fb5744bb3c"}
Jul 07 14:09:34.345817 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345770    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"89bfe88758b65589538fedf4ab480eace9eee2cc26ca7622c410b4b7349d21cb"}
Jul 07 14:09:34.345817 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345806    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"e2577ca155d33562f935cdd6429e92e0b3b6c77ca2161328ca54475f6e5dd85a"}
Jul 07 14:09:34.345879 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345834    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerDied","Data":"1a068dae809b3c713d45525b595e330573543556244d83f1699ce65975268866"}
Jul 07 14:09:34.345879 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:34.345859    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"9c2230b27fd6aa342fe3722799b26be65b9b4a14dbfd9c1aac24f586e7c3f2a3"}
Jul 07 14:09:35.236038 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:35.236004    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:35.236200 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:35.236101    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:35.236200 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:35.236103    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:35.236200 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:35.236110    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:35.236342 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:35.236218    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:35.236342 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:35.236308    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:36.351036 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:36.351011    2570 generic.go:356] "Generic (PLEG): container finished" podID="f3888bea-77ca-4f33-9126-f1673c37beb9" containerID="c9a26d7daef81d3f57faaafba62e052055c19f880f2020738619899d2b57a426" exitCode=0
Jul 07 14:09:36.351514 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:36.351083    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerDied","Data":"c9a26d7daef81d3f57faaafba62e052055c19f880f2020738619899d2b57a426"}
Jul 07 14:09:36.353784 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:36.353768    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:09:36.354212 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:36.354119    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"29ab25ff012d3988e3167e7df7ccba552542f1a6a2e32b308c4ad10c94a11755"}
Jul 07 14:09:37.236034 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:37.236006    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:37.236160 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:37.236030    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:37.236160 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:37.236030    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:37.236160 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:37.236109    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:37.236316 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:37.236205    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:37.236316 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:37.236288    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:37.358007 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:37.357985    2570 generic.go:356] "Generic (PLEG): container finished" podID="f3888bea-77ca-4f33-9126-f1673c37beb9" containerID="a18b016179ddf43380c0a4feaafcc7fbfbca50f704e209761b3f549b1e4cb0df" exitCode=0
Jul 07 14:09:37.358263 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:37.358016    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerDied","Data":"a18b016179ddf43380c0a4feaafcc7fbfbca50f704e209761b3f549b1e4cb0df"}
Jul 07 14:09:38.363366 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.363215    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:09:38.363733 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.363648    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"85b8b10e0fa76992bd90c91bd7b3065d7013ac2522f6795d4757537b1747eccc"}
Jul 07 14:09:38.363953 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.363935    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:38.364021 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.364002    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:38.364110 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.364092    2570 scope.go:117] "RemoveContainer" containerID="1a068dae809b3c713d45525b595e330573543556244d83f1699ce65975268866"
Jul 07 14:09:38.379576 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.379559    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:38.926050 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.926024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:38.926209 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:38.926057    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:38.926209 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:38.926145    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:38.926209 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:38.926161    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:38.926209 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:38.926208    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:10:10.926194926 +0000 UTC m=+65.215104002 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:09:38.926352 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:38.926221    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret podName:7b887d71-ef48-4b43-b8d2-700d180d40ff nodeName:}" failed. No retries permitted until 2026-07-07 14:10:10.926215096 +0000 UTC m=+65.215124185 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret") pod "global-pull-secret-syncer-cn8n9" (UID: "7b887d71-ef48-4b43-b8d2-700d180d40ff") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:09:39.026760 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.026733    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:39.026864 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.026849    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:09:39.026864 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.026863    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:09:39.026945 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.026871    2570 projected.go:196] Error preparing data for projected volume kube-api-access-6bf7d for pod openshift-network-diagnostics/network-check-target-v6sph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:39.026945 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.026908    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d podName:ad127895-5cdb-4412-81f3-f6ef235d8c7b nodeName:}" failed. No retries permitted until 2026-07-07 14:10:11.026897126 +0000 UTC m=+65.315806203 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6bf7d" (UniqueName: "kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d") pod "network-check-target-v6sph" (UID: "ad127895-5cdb-4412-81f3-f6ef235d8c7b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:09:39.236524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.236446    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:39.236524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.236491    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:39.236524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.236491    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:39.236732 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.236561    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:39.236732 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.236667    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:39.236803 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:39.236743    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:39.369300 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.369273    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:09:39.369700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.369553    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" event={"ID":"5979d983-b592-460c-a540-bf0049e7071e","Type":"ContainerStarted","Data":"2eb20c00baf1f0d472be331f8e7399ff4dd099b87b6b0373c3ae6e99d1b7c16e"}
Jul 07 14:09:39.369879 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.369861    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:39.383692 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.383670    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:09:39.402959 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:39.402709    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mns74" podStartSLOduration=7.69266157 podStartE2EDuration="33.402691567s" totalImagesPullingTime="25.710029997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-07 14:09:39.40129571 +0000 UTC m=+33.690204807" watchObservedRunningTime="2026-07-07 14:09:39.402691567 +0000 UTC m=+33.691600665"
Jul 07 14:09:40.258387 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:40.258358    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cn8n9"]
Jul 07 14:09:40.258555 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:40.258472    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:40.258596 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:40.258552    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:40.268047 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:40.268012    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x8nq2"]
Jul 07 14:09:40.268252 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:40.268140    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:40.268329 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:40.268287    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:40.268808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:40.268784    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v6sph"]
Jul 07 14:09:40.268894 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:40.268882    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:40.268995 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:40.268964    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:42.236472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:42.236305    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:42.236754 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:42.236305    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:42.236754 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:42.236565    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:42.236754 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:42.236336    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:42.236754 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:42.236609    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:42.236754 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:42.236711    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:44.236043 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:44.236009    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:44.236451 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:44.236020    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:44.236451 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:44.236137    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:44.236451 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:44.236019    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:44.236451 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:44.236215    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:44.236451 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:44.236257    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:46.237544 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.237521    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:46.238010 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:46.237592    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cn8n9" podUID="7b887d71-ef48-4b43-b8d2-700d180d40ff"
Jul 07 14:09:46.238010 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.237679    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:46.238010 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:46.237770    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:09:46.238010 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.237785    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:46.238010 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:46.237836    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v6sph" podUID="ad127895-5cdb-4412-81f3-f6ef235d8c7b"
Jul 07 14:09:46.386874 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.386851    2570 generic.go:356] "Generic (PLEG): container finished" podID="f3888bea-77ca-4f33-9126-f1673c37beb9" containerID="4a881a8dcf739018401488ca0d9a5700d236fcbb5a22092dfe48facc474afa94" exitCode=0
Jul 07 14:09:46.386967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.386889    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerDied","Data":"4a881a8dcf739018401488ca0d9a5700d236fcbb5a22092dfe48facc474afa94"}
Jul 07 14:09:46.451745 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.451725    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-145.ec2.internal" event="NodeReady"
Jul 07 14:09:46.451848 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.451836    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 14:09:46.500979 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.500925    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-95wtb"]
Jul 07 14:09:46.515018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.514996    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vshns"]
Jul 07 14:09:46.515137 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.515093    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.517946 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.517772    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2225x\""
Jul 07 14:09:46.518542 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.518376    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 14:09:46.518637 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.518574    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 14:09:46.535372 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.535343    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-95wtb"]
Jul 07 14:09:46.535458 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.535397    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vshns"]
Jul 07 14:09:46.535458 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.535351    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.538660 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.538642    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 14:09:46.539004 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.538983    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ppbqm\""
Jul 07 14:09:46.539394 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.539379    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 14:09:46.539483 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.539451    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 14:09:46.688970 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.688947    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0588c337-000a-4777-bfeb-67c419ebd711-config-volume\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.689070 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.688974    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.689070 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.688990    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5-cert\") pod \"ingress-canary-vshns\" (UID: \"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5\") " pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.689070 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.689007    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjqhx\" (UniqueName: \"kubernetes.io/projected/0588c337-000a-4777-bfeb-67c419ebd711-kube-api-access-sjqhx\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.689070 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.689029    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0588c337-000a-4777-bfeb-67c419ebd711-tmp-dir\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.689263 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.689162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcrr8\" (UniqueName: \"kubernetes.io/projected/b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5-kube-api-access-jcrr8\") pod \"ingress-canary-vshns\" (UID: \"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5\") " pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.790343 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790278    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcrr8\" (UniqueName: \"kubernetes.io/projected/b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5-kube-api-access-jcrr8\") pod \"ingress-canary-vshns\" (UID: \"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5\") " pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.790343 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790318    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0588c337-000a-4777-bfeb-67c419ebd711-config-volume\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.790495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.790495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790367    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5-cert\") pod \"ingress-canary-vshns\" (UID: \"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5\") " pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.790495 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:46.790459    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:09:46.790638 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:46.790529    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:47.290507673 +0000 UTC m=+41.579416795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:09:46.790638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790577    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjqhx\" (UniqueName: \"kubernetes.io/projected/0588c337-000a-4777-bfeb-67c419ebd711-kube-api-access-sjqhx\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.790638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790627    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0588c337-000a-4777-bfeb-67c419ebd711-tmp-dir\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.790913 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790893    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0588c337-000a-4777-bfeb-67c419ebd711-tmp-dir\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.790965 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.790931    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0588c337-000a-4777-bfeb-67c419ebd711-config-volume\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.794941 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.794914    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5-cert\") pod \"ingress-canary-vshns\" (UID: \"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5\") " pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.800240 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.800215    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjqhx\" (UniqueName: \"kubernetes.io/projected/0588c337-000a-4777-bfeb-67c419ebd711-kube-api-access-sjqhx\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:46.801290 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.801267    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcrr8\" (UniqueName: \"kubernetes.io/projected/b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5-kube-api-access-jcrr8\") pod \"ingress-canary-vshns\" (UID: \"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5\") " pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:46.843375 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:46.843353    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vshns"
Jul 07 14:09:47.031516 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:47.031385    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vshns"]
Jul 07 14:09:47.033646 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:09:47.033623    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c45f9e_7791_4b7c_b005_c9dbd2fd48b5.slice/crio-f38abab69c61a529299a63cc2caeca9473d054c5bb1fd0d990dcc28ce3d0d942 WatchSource:0}: Error finding container f38abab69c61a529299a63cc2caeca9473d054c5bb1fd0d990dcc28ce3d0d942: Status 404 returned error can't find the container with id f38abab69c61a529299a63cc2caeca9473d054c5bb1fd0d990dcc28ce3d0d942
Jul 07 14:09:47.295305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:47.295275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:47.295781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:47.295412    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:09:47.295781 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:47.295485    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:48.295465025 +0000 UTC m=+42.584374103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:09:47.390824 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:47.390790    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vshns" event={"ID":"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5","Type":"ContainerStarted","Data":"f38abab69c61a529299a63cc2caeca9473d054c5bb1fd0d990dcc28ce3d0d942"}
Jul 07 14:09:47.394035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:47.394011    2570 generic.go:356] "Generic (PLEG): container finished" podID="f3888bea-77ca-4f33-9126-f1673c37beb9" containerID="e22365b3a9e403110ae365b063fe3405cd706d816e8afcbfba519506954e5140" exitCode=0
Jul 07 14:09:47.394131 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:47.394050    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerDied","Data":"e22365b3a9e403110ae365b063fe3405cd706d816e8afcbfba519506954e5140"}
Jul 07 14:09:48.235873 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.235837    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:09:48.236035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.235836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:09:48.236035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.235964    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:09:48.238616 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.238592    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:09:48.239214 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.239197    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:09:48.239349 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.239246    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x2hv7\""
Jul 07 14:09:48.239463 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.239443    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:09:48.239560 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.239466    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lp7v4\""
Jul 07 14:09:48.239560 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.239506    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:09:48.302916 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.302869    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:48.303376 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:48.302981    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:09:48.303376 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:48.303046    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:50.303033102 +0000 UTC m=+44.591942178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:09:48.400481 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.400449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" event={"ID":"f3888bea-77ca-4f33-9126-f1673c37beb9","Type":"ContainerStarted","Data":"085a5c38c6ec38a6292a7452097415937ef218d5ce1d4d4bef6a0d2ca9f2b118"}
Jul 07 14:09:48.426950 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:48.426907    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jdw9x" podStartSLOduration=11.147939536 podStartE2EDuration="42.426893408s" totalImagesPullingTime="31.278953872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:48.425579896 +0000 UTC m=+42.714488985" watchObservedRunningTime="2026-07-07 14:09:48.426893408 +0000 UTC m=+42.715802507"
Jul 07 14:09:50.320217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:50.320038    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:50.320217 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:50.320209    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:09:50.320821 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:50.320285    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:09:54.320267148 +0000 UTC m=+48.609176237 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:09:50.406299 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:50.406270    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vshns" event={"ID":"b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5","Type":"ContainerStarted","Data":"fd9838ad5c70215df0f050fbbb4fc322b535f0374c9ea168a9c25f49154e9b8f"}
Jul 07 14:09:50.427212 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:50.427173    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vshns" podStartSLOduration=1.900717932 podStartE2EDuration="4.427138248s" totalImagesPullingTime="2.526420316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:09:50.426085175 +0000 UTC m=+44.714994273" watchObservedRunningTime="2026-07-07 14:09:50.427138248 +0000 UTC m=+44.716047346"
Jul 07 14:09:54.346938 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:09:54.346897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:09:54.347309 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:54.347036    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:09:54.347309 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:09:54.347106    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:10:02.347088383 +0000 UTC m=+56.635997461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:10:02.393853 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:02.393819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:10:02.394367 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:02.393960    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:10:02.394367 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:02.394014    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:10:18.39400031 +0000 UTC m=+72.682909386 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:10:10.938420 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:10.938388    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:10:10.938420 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:10.938425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:10:10.941284 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:10.941261    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:10:10.941336 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:10.941298    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:10:10.948981 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:10.948963    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 14:10:10.949041 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:10.949015    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:11:14.948999808 +0000 UTC m=+129.237908885 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : secret "metrics-daemon-secret" not found
Jul 07 14:10:10.952474 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:10.952454    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7b887d71-ef48-4b43-b8d2-700d180d40ff-original-pull-secret\") pod \"global-pull-secret-syncer-cn8n9\" (UID: \"7b887d71-ef48-4b43-b8d2-700d180d40ff\") " pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:10:11.038986 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.038955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:10:11.041560 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.041545    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:10:11.048738 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.048713    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cn8n9"
Jul 07 14:10:11.052905 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.052888    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:10:11.062357 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.062337    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bf7d\" (UniqueName: \"kubernetes.io/projected/ad127895-5cdb-4412-81f3-f6ef235d8c7b-kube-api-access-6bf7d\") pod \"network-check-target-v6sph\" (UID: \"ad127895-5cdb-4412-81f3-f6ef235d8c7b\") " pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:10:11.081803 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.081776    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lp7v4\""
Jul 07 14:10:11.087670 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.087650    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:10:11.177510 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.177482    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cn8n9"]
Jul 07 14:10:11.181584 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:10:11.181558    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b887d71_ef48_4b43_b8d2_700d180d40ff.slice/crio-0963e859e3287c7ed3cf91a719625a6ae7c7c25515eeacbac9c6705ff9bc0810 WatchSource:0}: Error finding container 0963e859e3287c7ed3cf91a719625a6ae7c7c25515eeacbac9c6705ff9bc0810: Status 404 returned error can't find the container with id 0963e859e3287c7ed3cf91a719625a6ae7c7c25515eeacbac9c6705ff9bc0810
Jul 07 14:10:11.215560 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.215507    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v6sph"]
Jul 07 14:10:11.218578 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:10:11.218553    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad127895_5cdb_4412_81f3_f6ef235d8c7b.slice/crio-1d0b4cd6ed1b74ef0c0a805ab844c7e51e0b7e43ba4f07d2b2455472b61b0f70 WatchSource:0}: Error finding container 1d0b4cd6ed1b74ef0c0a805ab844c7e51e0b7e43ba4f07d2b2455472b61b0f70: Status 404 returned error can't find the container with id 1d0b4cd6ed1b74ef0c0a805ab844c7e51e0b7e43ba4f07d2b2455472b61b0f70
Jul 07 14:10:11.384241 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.384220    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mns74"
Jul 07 14:10:11.449433 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.449403    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v6sph" event={"ID":"ad127895-5cdb-4412-81f3-f6ef235d8c7b","Type":"ContainerStarted","Data":"1d0b4cd6ed1b74ef0c0a805ab844c7e51e0b7e43ba4f07d2b2455472b61b0f70"}
Jul 07 14:10:11.450864 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:11.450838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cn8n9" event={"ID":"7b887d71-ef48-4b43-b8d2-700d180d40ff","Type":"ContainerStarted","Data":"0963e859e3287c7ed3cf91a719625a6ae7c7c25515eeacbac9c6705ff9bc0810"}
Jul 07 14:10:17.471218 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:17.471178    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v6sph" event={"ID":"ad127895-5cdb-4412-81f3-f6ef235d8c7b","Type":"ContainerStarted","Data":"94505032fedca520e0db2635e52b9f3c051c41db591238dbf3e6742270d360fb"}
Jul 07 14:10:17.471635 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:17.471323    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:10:17.472495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:17.472471    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cn8n9" event={"ID":"7b887d71-ef48-4b43-b8d2-700d180d40ff","Type":"ContainerStarted","Data":"c4ae5e4c45e55d5849434450bcb0b13ccf46be3e127ea82aa77f72c9980724a9"}
Jul 07 14:10:17.487683 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:17.487641    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v6sph" podStartSLOduration=66.00398921 podStartE2EDuration="1m11.48762515s" totalImagesPullingTime="5.48363594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:10:17.486966904 +0000 UTC m=+71.775876017" watchObservedRunningTime="2026-07-07 14:10:17.48762515 +0000 UTC m=+71.776534284"
Jul 07 14:10:17.502432 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:17.502386    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cn8n9" podStartSLOduration=65.971277075 podStartE2EDuration="1m11.50237144s" totalImagesPullingTime="5.531094365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:10:17.502265457 +0000 UTC m=+71.791174555" watchObservedRunningTime="2026-07-07 14:10:17.50237144 +0000 UTC m=+71.791280539"
Jul 07 14:10:18.490068 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:18.490031    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:10:18.490427 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:18.490126    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:10:18.490427 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:18.490207    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:10:50.490187075 +0000 UTC m=+104.779096168 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:10:25.617345 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.617296    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"]
Jul 07 14:10:25.623548 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.623523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.627191 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.627169    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 14:10:25.628229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.628201    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 14:10:25.628346 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.628251    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 14:10:25.628346 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.628273    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4qcff\""
Jul 07 14:10:25.628346 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.628251    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 14:10:25.632313 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.632284    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"]
Jul 07 14:10:25.733404 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.733374    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pld2\" (UniqueName: \"kubernetes.io/projected/38efc826-5955-472d-9e07-e25eb4e842af-kube-api-access-4pld2\") pod \"managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9\" (UID: \"38efc826-5955-472d-9e07-e25eb4e842af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.733404 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.733406    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38efc826-5955-472d-9e07-e25eb4e842af-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9\" (UID: \"38efc826-5955-472d-9e07-e25eb4e842af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.834517 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.834483    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pld2\" (UniqueName: \"kubernetes.io/projected/38efc826-5955-472d-9e07-e25eb4e842af-kube-api-access-4pld2\") pod \"managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9\" (UID: \"38efc826-5955-472d-9e07-e25eb4e842af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.834517 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.834523    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38efc826-5955-472d-9e07-e25eb4e842af-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9\" (UID: \"38efc826-5955-472d-9e07-e25eb4e842af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.837044 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.837023    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38efc826-5955-472d-9e07-e25eb4e842af-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9\" (UID: \"38efc826-5955-472d-9e07-e25eb4e842af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.847033 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.847005    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pld2\" (UniqueName: \"kubernetes.io/projected/38efc826-5955-472d-9e07-e25eb4e842af-kube-api-access-4pld2\") pod \"managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9\" (UID: \"38efc826-5955-472d-9e07-e25eb4e842af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:25.942583 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:25.942523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"
Jul 07 14:10:26.069066 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:26.069030    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9"]
Jul 07 14:10:26.072358 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:10:26.072329    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38efc826_5955_472d_9e07_e25eb4e842af.slice/crio-6ab1438d85ec58ec3550d8344427f252ccea16ea1ca3b41ea77fff9ee51ffd8f WatchSource:0}: Error finding container 6ab1438d85ec58ec3550d8344427f252ccea16ea1ca3b41ea77fff9ee51ffd8f: Status 404 returned error can't find the container with id 6ab1438d85ec58ec3550d8344427f252ccea16ea1ca3b41ea77fff9ee51ffd8f
Jul 07 14:10:26.493930 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:26.493903    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9" event={"ID":"38efc826-5955-472d-9e07-e25eb4e842af","Type":"ContainerStarted","Data":"6ab1438d85ec58ec3550d8344427f252ccea16ea1ca3b41ea77fff9ee51ffd8f"}
Jul 07 14:10:29.503324 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:29.503288    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9" event={"ID":"38efc826-5955-472d-9e07-e25eb4e842af","Type":"ContainerStarted","Data":"35ca3a8d36e68af34b3181f3fbb4ebd19322362ec9fc2dbfd6bb112fc2879b27"}
Jul 07 14:10:29.531464 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:29.531414    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9" podStartSLOduration=1.872652921 podStartE2EDuration="4.531398511s" totalImagesPullingTime="2.65874559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:10:29.530106152 +0000 UTC m=+83.819015270" watchObservedRunningTime="2026-07-07 14:10:29.531398511 +0000 UTC m=+83.820307609"
Jul 07 14:10:48.476410 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:48.476374    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v6sph"
Jul 07 14:10:50.492504 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:10:50.492472    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:10:50.492872 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:50.492587    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:10:50.492872 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:10:50.492643    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:11:54.492628141 +0000 UTC m=+168.781537217 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:11:15.048110 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:11:15.048072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:11:15.048728 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:11:15.048269    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 14:11:15.048728 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:11:15.048371    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:17.048348934 +0000 UTC m=+251.337258050 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : secret "metrics-daemon-secret" not found
Jul 07 14:11:49.526139 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:11:49.526096    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-95wtb" podUID="0588c337-000a-4777-bfeb-67c419ebd711"
Jul 07 14:11:49.683361 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:11:49.683334    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95wtb"
Jul 07 14:11:51.254743 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:11:51.254704    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:11:54.496642 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:11:54.496601    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:11:54.497035 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:11:54.496708    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:11:54.497035 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:11:54.496773    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls podName:0588c337-000a-4777-bfeb-67c419ebd711 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:56.496758486 +0000 UTC m=+290.785667562 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls") pod "dns-default-95wtb" (UID: "0588c337-000a-4777-bfeb-67c419ebd711") : secret "dns-default-metrics-tls" not found
Jul 07 14:12:06.237241 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:12:06.237160    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:12:29.773139 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:12:29.773094    2570 generic.go:356] "Generic (PLEG): container finished" podID="38efc826-5955-472d-9e07-e25eb4e842af" containerID="35ca3a8d36e68af34b3181f3fbb4ebd19322362ec9fc2dbfd6bb112fc2879b27" exitCode=255
Jul 07 14:12:29.773513 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:12:29.773174    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9" event={"ID":"38efc826-5955-472d-9e07-e25eb4e842af","Type":"ContainerDied","Data":"35ca3a8d36e68af34b3181f3fbb4ebd19322362ec9fc2dbfd6bb112fc2879b27"}
Jul 07 14:12:29.779189 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:12:29.779171    2570 scope.go:117] "RemoveContainer" containerID="35ca3a8d36e68af34b3181f3fbb4ebd19322362ec9fc2dbfd6bb112fc2879b27"
Jul 07 14:12:30.778625 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:12:30.778592    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8575c6fbcd-m4qg9" event={"ID":"38efc826-5955-472d-9e07-e25eb4e842af","Type":"ContainerStarted","Data":"43daf03405321534f6b9730958439dfd74e4e9e13d55b606f27bcd1df65c84e2"}
Jul 07 14:13:17.107970 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:17.107915    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:13:17.108515 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:17.108065    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 14:13:17.108515 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:17.108135    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs podName:015c7f85-80c7-42cf-8c77-6f22642344c1 nodeName:}" failed. No retries permitted until 2026-07-07 14:15:19.10811699 +0000 UTC m=+373.397026070 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs") pod "network-metrics-daemon-x8nq2" (UID: "015c7f85-80c7-42cf-8c77-6f22642344c1") : secret "metrics-daemon-secret" not found
Jul 07 14:13:37.728316 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.728284    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d9ff8d777-5fj9n"]
Jul 07 14:13:37.731185 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.731141    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"]
Jul 07 14:13:37.731315 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.731271    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.734975 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.734955    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:37.735514 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.735493    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 14:13:37.735634 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.735589    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6rbz7\""
Jul 07 14:13:37.736503 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.736483    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 14:13:37.740082 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.740062    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 14:13:37.744458 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.744439    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 14:13:37.744686 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.744669    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 14:13:37.744762 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.744728    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:13:37.744894 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.744880    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tz789\""
Jul 07 14:13:37.746411 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.746368    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 14:13:37.760727 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.760702    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d9ff8d777-5fj9n"]
Jul 07 14:13:37.762578 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.762558    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"]
Jul 07 14:13:37.839071 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.839044    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"]
Jul 07 14:13:37.839852 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.839831    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-certificates\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.839940 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.839858    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-trusted-ca\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.839940 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.839875    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-installation-pull-secrets\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.839940 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.839891    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkfkb\" (UniqueName: \"kubernetes.io/projected/8b785d9c-7605-4b26-b4d5-ef63167b72d6-kube-api-access-qkfkb\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:37.840098 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.839966    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:37.840098 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.840012    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-image-registry-private-configuration\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.840098 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.840031    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvjx5\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-kube-api-access-fvjx5\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.840098 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.840061    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.840098 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.840075    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8190923-ba50-4711-92ea-fd515ca6b73b-ca-trust-extracted\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.840367 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.840100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-bound-sa-token\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.841726 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.841713    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74ff99db5f-jzfph"]
Jul 07 14:13:37.841852 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.841836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:37.844360 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.844344    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"]
Jul 07 14:13:37.844457 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.844444    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:37.846977 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.846961    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:37.848594 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.848578    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 07 14:13:37.848748 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.848731    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 07 14:13:37.848855 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.848835    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 07 14:13:37.848998 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.848984    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zpbwd\""
Jul 07 14:13:37.849263 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.849248    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 14:13:37.849333 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.849274    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-znndp\""
Jul 07 14:13:37.849592 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.849579    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 14:13:37.849650 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.849638    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:13:37.850049 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.850038    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 14:13:37.852236 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852224    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 14:13:37.852674 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852613    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 07 14:13:37.852674 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852660    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kc898\""
Jul 07 14:13:37.852788 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852687    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 07 14:13:37.852788 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852702    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 14:13:37.852788 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852784    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 07 14:13:37.852978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.852864    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 07 14:13:37.855582 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.855567    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:13:37.860475 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.860457    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"]
Jul 07 14:13:37.867614 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.867596    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"]
Jul 07 14:13:37.868311 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.868289    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74ff99db5f-jzfph"]
Jul 07 14:13:37.941017 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.940992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:37.941120 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941029    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a36602f-4dd6-4ca8-aff4-646066eaf071-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:37.941120 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.941120 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941066    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzslt\" (UniqueName: \"kubernetes.io/projected/f6756644-141f-4d50-84a0-7a342e9b80e1-kube-api-access-gzslt\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:37.941120 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a36602f-4dd6-4ca8-aff4-646066eaf071-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkfkb\" (UniqueName: \"kubernetes.io/projected/8b785d9c-7605-4b26-b4d5-ef63167b72d6-kube-api-access-qkfkb\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:37.941222    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941231    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:37.941238    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9ff8d777-5fj9n: secret "image-registry-tls" not found
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941252    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gljpx\" (UniqueName: \"kubernetes.io/projected/aec2ddc5-6572-4ebe-a4f8-a42897d65002-kube-api-access-gljpx\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941269    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zncdd\" (UniqueName: \"kubernetes.io/projected/4a36602f-4dd6-4ca8-aff4-646066eaf071-kube-api-access-zncdd\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6756644-141f-4d50-84a0-7a342e9b80e1-config\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:37.941332 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:37.941306    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls podName:e8190923-ba50-4711-92ea-fd515ca6b73b nodeName:}" failed. No retries permitted until 2026-07-07 14:13:38.441290818 +0000 UTC m=+272.730199897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls") pod "image-registry-7d9ff8d777-5fj9n" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b") : secret "image-registry-tls" not found
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941481    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvjx5\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-kube-api-access-fvjx5\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:37.941559    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-image-registry-private-configuration\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:37.941607    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls podName:8b785d9c-7605-4b26-b4d5-ef63167b72d6 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:38.44159592 +0000 UTC m=+272.730505000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht89x" (UID: "8b785d9c-7605-4b26-b4d5-ef63167b72d6") : secret "samples-operator-tls" not found
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941621    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-stats-auth\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:37.941704 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941648    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-trusted-ca\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941736    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-bound-sa-token\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941758    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-certificates\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941774    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6756644-141f-4d50-84a0-7a342e9b80e1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:37.942018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941799    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8190923-ba50-4711-92ea-fd515ca6b73b-ca-trust-extracted\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941816    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-default-certificate\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:37.942018 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.941834    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-installation-pull-secrets\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942298 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.942122    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8190923-ba50-4711-92ea-fd515ca6b73b-ca-trust-extracted\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942298 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.942284    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-certificates\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.942605 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.942589    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-trusted-ca\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.943955 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.943932    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-image-registry-private-configuration\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.944076 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.943982    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-installation-pull-secrets\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.966124 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.966094    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvjx5\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-kube-api-access-fvjx5\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:37.966220 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.966138    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkfkb\" (UniqueName: \"kubernetes.io/projected/8b785d9c-7605-4b26-b4d5-ef63167b72d6-kube-api-access-qkfkb\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:37.967724 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:37.967702    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-bound-sa-token\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:38.042777 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042731    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6756644-141f-4d50-84a0-7a342e9b80e1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.042777 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042765    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-default-certificate\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.042889 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042786    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.042889 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a36602f-4dd6-4ca8-aff4-646066eaf071-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.042959 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042910    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzslt\" (UniqueName: \"kubernetes.io/projected/f6756644-141f-4d50-84a0-7a342e9b80e1-kube-api-access-gzslt\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.042959 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042924    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a36602f-4dd6-4ca8-aff4-646066eaf071-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.042959 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042942    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.042959 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.042948    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:13:38.043182 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042960    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gljpx\" (UniqueName: \"kubernetes.io/projected/aec2ddc5-6572-4ebe-a4f8-a42897d65002-kube-api-access-gljpx\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.043182 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.042976    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zncdd\" (UniqueName: \"kubernetes.io/projected/4a36602f-4dd6-4ca8-aff4-646066eaf071-kube-api-access-zncdd\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.043182 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.043010    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:38.542993205 +0000 UTC m=+272.831902304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : secret "router-metrics-certs-default" not found
Jul 07 14:13:38.043182 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.043033    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6756644-141f-4d50-84a0-7a342e9b80e1-config\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.043182 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.043090    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-stats-auth\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.043421 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.043192    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:38.543176459 +0000 UTC m=+272.832085541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : configmap references non-existent config key: service-ca.crt
Jul 07 14:13:38.043587 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.043553    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a36602f-4dd6-4ca8-aff4-646066eaf071-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.043755 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.043738    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6756644-141f-4d50-84a0-7a342e9b80e1-config\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.044885 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.044867    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a36602f-4dd6-4ca8-aff4-646066eaf071-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.045285 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.045265    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-stats-auth\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.045391 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.045341    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6756644-141f-4d50-84a0-7a342e9b80e1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.045451 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.045413    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-default-certificate\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.068626 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.068604    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzslt\" (UniqueName: \"kubernetes.io/projected/f6756644-141f-4d50-84a0-7a342e9b80e1-kube-api-access-gzslt\") pod \"service-ca-operator-5d46d9ffd4-nkmpx\" (UID: \"f6756644-141f-4d50-84a0-7a342e9b80e1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.070474 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.070454    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gljpx\" (UniqueName: \"kubernetes.io/projected/aec2ddc5-6572-4ebe-a4f8-a42897d65002-kube-api-access-gljpx\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.071969 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.071952    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zncdd\" (UniqueName: \"kubernetes.io/projected/4a36602f-4dd6-4ca8-aff4-646066eaf071-kube-api-access-zncdd\") pod \"kube-storage-version-migrator-operator-846bf668f7-m7z4g\" (UID: \"4a36602f-4dd6-4ca8-aff4-646066eaf071\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.151360 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.151339    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"
Jul 07 14:13:38.161618 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.161601    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"
Jul 07 14:13:38.288792 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.288736    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g"]
Jul 07 14:13:38.292591 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:13:38.292569    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a36602f_4dd6_4ca8_aff4_646066eaf071.slice/crio-7b587fdb47040f45f73ff1cccd7073c1f4fd75edee2f95821a1dbf283f09ec74 WatchSource:0}: Error finding container 7b587fdb47040f45f73ff1cccd7073c1f4fd75edee2f95821a1dbf283f09ec74: Status 404 returned error can't find the container with id 7b587fdb47040f45f73ff1cccd7073c1f4fd75edee2f95821a1dbf283f09ec74
Jul 07 14:13:38.307468 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.307444    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx"]
Jul 07 14:13:38.310005 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:13:38.309985    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6756644_141f_4d50_84a0_7a342e9b80e1.slice/crio-357b6dae836bae98978614e7789d0478e00621302a91bba4d40805d9e9448a14 WatchSource:0}: Error finding container 357b6dae836bae98978614e7789d0478e00621302a91bba4d40805d9e9448a14: Status 404 returned error can't find the container with id 357b6dae836bae98978614e7789d0478e00621302a91bba4d40805d9e9448a14
Jul 07 14:13:38.447355 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.447329    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:38.447460 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.447388    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:38.447520 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.447471    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:13:38.447520 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.447486    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9ff8d777-5fj9n: secret "image-registry-tls" not found
Jul 07 14:13:38.447520 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.447505    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:13:38.447642 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.447529    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls podName:e8190923-ba50-4711-92ea-fd515ca6b73b nodeName:}" failed. No retries permitted until 2026-07-07 14:13:39.447514445 +0000 UTC m=+273.736423521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls") pod "image-registry-7d9ff8d777-5fj9n" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b") : secret "image-registry-tls" not found
Jul 07 14:13:38.447642 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.447556    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls podName:8b785d9c-7605-4b26-b4d5-ef63167b72d6 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:39.447539615 +0000 UTC m=+273.736448720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht89x" (UID: "8b785d9c-7605-4b26-b4d5-ef63167b72d6") : secret "samples-operator-tls" not found
Jul 07 14:13:38.548061 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.548007    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.548142 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.548064    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:38.548208 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.548141    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:13:38.548208 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.548187    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:39.548173651 +0000 UTC m=+273.837082732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : configmap references non-existent config key: service-ca.crt
Jul 07 14:13:38.548208 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:38.548205    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:39.548198915 +0000 UTC m=+273.837107991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : secret "router-metrics-certs-default" not found
Jul 07 14:13:38.978482 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.978453    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx" event={"ID":"f6756644-141f-4d50-84a0-7a342e9b80e1","Type":"ContainerStarted","Data":"357b6dae836bae98978614e7789d0478e00621302a91bba4d40805d9e9448a14"}
Jul 07 14:13:38.979657 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:38.979630    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g" event={"ID":"4a36602f-4dd6-4ca8-aff4-646066eaf071","Type":"ContainerStarted","Data":"7b587fdb47040f45f73ff1cccd7073c1f4fd75edee2f95821a1dbf283f09ec74"}
Jul 07 14:13:39.456408 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:39.456378    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:39.456580 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:39.456445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:39.456580 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.456495    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:13:39.456580 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.456515    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9ff8d777-5fj9n: secret "image-registry-tls" not found
Jul 07 14:13:39.456703 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.456581    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:13:39.456703 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.456583    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls podName:e8190923-ba50-4711-92ea-fd515ca6b73b nodeName:}" failed. No retries permitted until 2026-07-07 14:13:41.456564994 +0000 UTC m=+275.745474075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls") pod "image-registry-7d9ff8d777-5fj9n" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b") : secret "image-registry-tls" not found
Jul 07 14:13:39.456703 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.456636    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls podName:8b785d9c-7605-4b26-b4d5-ef63167b72d6 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:41.456623963 +0000 UTC m=+275.745533039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht89x" (UID: "8b785d9c-7605-4b26-b4d5-ef63167b72d6") : secret "samples-operator-tls" not found
Jul 07 14:13:39.557337 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:39.557313    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:39.557466 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:39.557375    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:39.557516 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.557461    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:13:39.557550 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.557516    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:41.557503861 +0000 UTC m=+275.846412950 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : secret "router-metrics-certs-default" not found
Jul 07 14:13:39.557550 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:39.557532    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:41.557524632 +0000 UTC m=+275.846433708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : configmap references non-existent config key: service-ca.crt
Jul 07 14:13:40.987840 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:40.987800    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx" event={"ID":"f6756644-141f-4d50-84a0-7a342e9b80e1","Type":"ContainerStarted","Data":"27e376a5b777bd823eac18bc8e875c17a1d629272568ee9dd883a4fc1cb6fbad"}
Jul 07 14:13:41.004762 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:41.004678    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx" podStartSLOduration=1.5663790450000001 podStartE2EDuration="4.004661599s" totalImagesPullingTime="2.438282554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:13:41.003755039 +0000 UTC m=+275.292664137" watchObservedRunningTime="2026-07-07 14:13:41.004661599 +0000 UTC m=+275.293570707"
Jul 07 14:13:41.473863 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:41.473839    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:41.474000 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:41.473879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:41.474000 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.473979    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:13:41.474087 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.473997    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:13:41.474087 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.474016    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9ff8d777-5fj9n: secret "image-registry-tls" not found
Jul 07 14:13:41.474087 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.474021    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls podName:8b785d9c-7605-4b26-b4d5-ef63167b72d6 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:45.474009295 +0000 UTC m=+279.762918371 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht89x" (UID: "8b785d9c-7605-4b26-b4d5-ef63167b72d6") : secret "samples-operator-tls" not found
Jul 07 14:13:41.474087 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.474061    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls podName:e8190923-ba50-4711-92ea-fd515ca6b73b nodeName:}" failed. No retries permitted until 2026-07-07 14:13:45.474046004 +0000 UTC m=+279.762955084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls") pod "image-registry-7d9ff8d777-5fj9n" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b") : secret "image-registry-tls" not found
Jul 07 14:13:41.575197 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:41.575125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:41.575305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:41.575204    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:41.575305 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.575236    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:13:41.575305 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.575273    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:45.575262557 +0000 UTC m=+279.864171633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : secret "router-metrics-certs-default" not found
Jul 07 14:13:41.575305 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:41.575303    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:45.575290209 +0000 UTC m=+279.864199285 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : configmap references non-existent config key: service-ca.crt
Jul 07 14:13:41.992173 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:41.992131    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g" event={"ID":"4a36602f-4dd6-4ca8-aff4-646066eaf071","Type":"ContainerStarted","Data":"6c96827ba38cf29006e5078cf4c2a138777abba6be0f9ce8cb2118ac4970aa62"}
Jul 07 14:13:42.010131 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.010093    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g" podStartSLOduration=1.994341363 podStartE2EDuration="5.010076778s" totalImagesPullingTime="3.015735415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:13:42.009131918 +0000 UTC m=+276.298041015" watchObservedRunningTime="2026-07-07 14:13:42.010076778 +0000 UTC m=+276.298985876"
Jul 07 14:13:42.885773 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.885738    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jtnbn"]
Jul 07 14:13:42.889058 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.889042    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:42.892006 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.891979    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xcd9z\""
Jul 07 14:13:42.892179 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.892026    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 14:13:42.892787 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.892763    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 14:13:42.892884 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.892865    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 14:13:42.895941 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.895926    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 14:13:42.900240 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.900222    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jtnbn"]
Jul 07 14:13:42.985583 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.985562    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-crio-socket\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:42.985687 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.985589    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:42.985687 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.985624    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45hll\" (UniqueName: \"kubernetes.io/projected/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-api-access-45hll\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:42.985687 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.985662    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:42.985687 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:42.985686    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-data-volume\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.086823 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.086802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45hll\" (UniqueName: \"kubernetes.io/projected/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-api-access-45hll\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.086859    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.086933    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-data-volume\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.086958    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.087014    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:43.586997529 +0000 UTC m=+277.875906609 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : configmap "kube-rbac-proxy" not found
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.087103    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-crio-socket\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.087139    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.087214    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-crio-socket\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087460 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.087290    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-data-volume\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.087460 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.087304    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:43.087460 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.087375    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:43.587356786 +0000 UTC m=+277.876265862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:43.098729 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.098705    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45hll\" (UniqueName: \"kubernetes.io/projected/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-api-access-45hll\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.591910 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.591882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.592065 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:43.591981    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:43.592065 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.592016    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:13:43.592195 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.592092    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:44.59207501 +0000 UTC m=+278.880984109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : configmap "kube-rbac-proxy" not found
Jul 07 14:13:43.592195 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.592130    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:43.592268 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:43.592237    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:44.592220538 +0000 UTC m=+278.881129616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:44.598059 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:44.598024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:44.598456 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:44.598093    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:44.598456 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:44.598163    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:13:44.598456 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:44.598213    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:44.598456 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:44.598238    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:46.598217764 +0000 UTC m=+280.887126858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : configmap "kube-rbac-proxy" not found
Jul 07 14:13:44.598456 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:44.598268    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:46.598255113 +0000 UTC m=+280.887164189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:45.195639 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.195615    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-q7sms"]
Jul 07 14:13:45.198711 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.198695    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.202104 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.202069    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 14:13:45.202517 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.202210    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 14:13:45.203294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.203272    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 14:13:45.203720 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.203592    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 14:13:45.204080 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.204056    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-q5cg8\""
Jul 07 14:13:45.209261 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.209238    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-q7sms"]
Jul 07 14:13:45.304699 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.304669    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d2461ccc-e3ec-4837-b0b9-90f461b455eb-signing-key\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.304821 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.304722    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d2461ccc-e3ec-4837-b0b9-90f461b455eb-signing-cabundle\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.304821 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.304748    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npx75\" (UniqueName: \"kubernetes.io/projected/d2461ccc-e3ec-4837-b0b9-90f461b455eb-kube-api-access-npx75\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.405286 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.405245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d2461ccc-e3ec-4837-b0b9-90f461b455eb-signing-cabundle\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.405409 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.405306    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npx75\" (UniqueName: \"kubernetes.io/projected/d2461ccc-e3ec-4837-b0b9-90f461b455eb-kube-api-access-npx75\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.405472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.405448    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d2461ccc-e3ec-4837-b0b9-90f461b455eb-signing-key\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.405901 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.405878    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d2461ccc-e3ec-4837-b0b9-90f461b455eb-signing-cabundle\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.407947 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.407920    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d2461ccc-e3ec-4837-b0b9-90f461b455eb-signing-key\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.414492 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.414465    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npx75\" (UniqueName: \"kubernetes.io/projected/d2461ccc-e3ec-4837-b0b9-90f461b455eb-kube-api-access-npx75\") pod \"service-ca-758f599c5-q7sms\" (UID: \"d2461ccc-e3ec-4837-b0b9-90f461b455eb\") " pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.506731 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.506666    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:45.506731 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.506720    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:45.506866 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.506797    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 14:13:45.506866 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.506806    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 14:13:45.506866 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.506827    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d9ff8d777-5fj9n: secret "image-registry-tls" not found
Jul 07 14:13:45.506977 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.506881    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls podName:e8190923-ba50-4711-92ea-fd515ca6b73b nodeName:}" failed. No retries permitted until 2026-07-07 14:13:53.506864595 +0000 UTC m=+287.795773725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls") pod "image-registry-7d9ff8d777-5fj9n" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b") : secret "image-registry-tls" not found
Jul 07 14:13:45.506977 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.506896    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls podName:8b785d9c-7605-4b26-b4d5-ef63167b72d6 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:53.506889693 +0000 UTC m=+287.795798769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht89x" (UID: "8b785d9c-7605-4b26-b4d5-ef63167b72d6") : secret "samples-operator-tls" not found
Jul 07 14:13:45.510675 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.510658    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-q7sms"
Jul 07 14:13:45.607863 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.607834    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:45.608264 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.607901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:45.608264 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.607997    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 14:13:45.608264 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.608061    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:53.608045138 +0000 UTC m=+287.896954214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : secret "router-metrics-certs-default" not found
Jul 07 14:13:45.608264 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:45.608076    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:53.608070283 +0000 UTC m=+287.896979359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : configmap references non-existent config key: service-ca.crt
Jul 07 14:13:45.656716 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.656692    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-q7sms"]
Jul 07 14:13:45.660107 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:13:45.660079    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2461ccc_e3ec_4837_b0b9_90f461b455eb.slice/crio-7fa542bfb7cb2612da5e561c4fd47219075462e6445cbad4e16ac3a7360e1c95 WatchSource:0}: Error finding container 7fa542bfb7cb2612da5e561c4fd47219075462e6445cbad4e16ac3a7360e1c95: Status 404 returned error can't find the container with id 7fa542bfb7cb2612da5e561c4fd47219075462e6445cbad4e16ac3a7360e1c95
Jul 07 14:13:45.675700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:45.675684    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6nzt4_32823771-b3e3-4806-a6eb-4e66616a5b48/dns-node-resolver/0.log"
Jul 07 14:13:46.005258 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:46.005228    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-q7sms" event={"ID":"d2461ccc-e3ec-4837-b0b9-90f461b455eb","Type":"ContainerStarted","Data":"ba2a03daa5348e18cdb068bf6c71d774c66fb9095b0321bd7b6b4d29497640cc"}
Jul 07 14:13:46.005382 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:46.005278    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-q7sms" event={"ID":"d2461ccc-e3ec-4837-b0b9-90f461b455eb","Type":"ContainerStarted","Data":"7fa542bfb7cb2612da5e561c4fd47219075462e6445cbad4e16ac3a7360e1c95"}
Jul 07 14:13:46.024557 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:46.024515    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-q7sms" podStartSLOduration=1.024500955 podStartE2EDuration="1.024500955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:13:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:13:46.024343913 +0000 UTC m=+280.313253022" watchObservedRunningTime="2026-07-07 14:13:46.024500955 +0000 UTC m=+280.313410052"
Jul 07 14:13:46.469757 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:46.469733    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-47k5s_64016df9-5267-45ef-bd4f-2b13535b77c4/node-ca/0.log"
Jul 07 14:13:46.615489 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:46.615465    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:46.615765 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:46.615523    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:46.615765 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:46.615596    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:13:46.615765 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:46.615601    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:46.615765 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:46.615639    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:50.61562631 +0000 UTC m=+284.904535386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : configmap "kube-rbac-proxy" not found
Jul 07 14:13:46.615765 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:46.615652    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:50.615645992 +0000 UTC m=+284.904555068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:47.268668 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:47.268649    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vshns_b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5/serve-healthcheck-canary/0.log"
Jul 07 14:13:48.271139 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:48.271113    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-m7z4g_4a36602f-4dd6-4ca8-aff4-646066eaf071/kube-storage-version-migrator-operator/0.log"
Jul 07 14:13:50.647164 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:50.647120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:50.647580 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:50.647217    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:50.647580 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:50.647242    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:13:50.647580 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:50.647305    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:58.647288639 +0000 UTC m=+292.936197728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : configmap "kube-rbac-proxy" not found
Jul 07 14:13:50.647580 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:50.647344    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:50.647580 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:50.647403    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls podName:ecc8e344-fafd-48aa-9d19-b5f168f0ec43 nodeName:}" failed. No retries permitted until 2026-07-07 14:13:58.6473871 +0000 UTC m=+292.936296180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jtnbn" (UID: "ecc8e344-fafd-48aa-9d19-b5f168f0ec43") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:13:52.684579 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:52.684548    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-95wtb" podUID="0588c337-000a-4777-bfeb-67c419ebd711"
Jul 07 14:13:53.023834 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.023765    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95wtb"
Jul 07 14:13:53.571627 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.571599    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:53.571780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.571643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:53.573957 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.573932    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8b785d9c-7605-4b26-b4d5-ef63167b72d6-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht89x\" (UID: \"8b785d9c-7605-4b26-b4d5-ef63167b72d6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:53.574071 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.573999    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"image-registry-7d9ff8d777-5fj9n\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") " pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:53.642978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.642951    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:53.650882 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.650863    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"
Jul 07 14:13:53.673374 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.673341    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:53.673516 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:13:53.673498    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle podName:aec2ddc5-6572-4ebe-a4f8-a42897d65002 nodeName:}" failed. No retries permitted until 2026-07-07 14:14:09.673476464 +0000 UTC m=+303.962385549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle") pod "router-default-74ff99db5f-jzfph" (UID: "aec2ddc5-6572-4ebe-a4f8-a42897d65002") : configmap references non-existent config key: service-ca.crt
Jul 07 14:13:53.673608 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.673582    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:53.675720 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.675698    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aec2ddc5-6572-4ebe-a4f8-a42897d65002-metrics-certs\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:13:53.781338 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.781312    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d9ff8d777-5fj9n"]
Jul 07 14:13:53.783705 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:13:53.783682    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8190923_ba50_4711_92ea_fd515ca6b73b.slice/crio-d4b351da8b1034cde350fbf0334d2956c76097326c7611a295caa8dad933b7fe WatchSource:0}: Error finding container d4b351da8b1034cde350fbf0334d2956c76097326c7611a295caa8dad933b7fe: Status 404 returned error can't find the container with id d4b351da8b1034cde350fbf0334d2956c76097326c7611a295caa8dad933b7fe
Jul 07 14:13:53.791612 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:53.791588    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x"]
Jul 07 14:13:54.028481 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:54.028446    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" event={"ID":"e8190923-ba50-4711-92ea-fd515ca6b73b","Type":"ContainerStarted","Data":"4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d"}
Jul 07 14:13:54.028648 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:54.028507    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" event={"ID":"e8190923-ba50-4711-92ea-fd515ca6b73b","Type":"ContainerStarted","Data":"d4b351da8b1034cde350fbf0334d2956c76097326c7611a295caa8dad933b7fe"}
Jul 07 14:13:54.028648 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:54.028562    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:13:54.029825 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:54.029802    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x" event={"ID":"8b785d9c-7605-4b26-b4d5-ef63167b72d6","Type":"ContainerStarted","Data":"dc57aac10ff5a83e06aee0ccde8acd3693eba07003fe8b91d30e8879231a4717"}
Jul 07 14:13:54.050664 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:54.050618    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" podStartSLOduration=17.050602159 podStartE2EDuration="17.050602159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:13:54.050423657 +0000 UTC m=+288.339332756" watchObservedRunningTime="2026-07-07 14:13:54.050602159 +0000 UTC m=+288.339511261"
Jul 07 14:13:56.038970 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:56.038945    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x" event={"ID":"8b785d9c-7605-4b26-b4d5-ef63167b72d6","Type":"ContainerStarted","Data":"3ecacd0764a27f1a658dd416510ae4ec121b9c28276d5c92d6108528deb1bb02"}
Jul 07 14:13:56.596169 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:56.596123    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:13:56.598192 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:56.598172    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0588c337-000a-4777-bfeb-67c419ebd711-metrics-tls\") pod \"dns-default-95wtb\" (UID: \"0588c337-000a-4777-bfeb-67c419ebd711\") " pod="openshift-dns/dns-default-95wtb"
Jul 07 14:13:56.626577 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:56.626552    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2225x\""
Jul 07 14:13:56.634620 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:56.634607    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95wtb"
Jul 07 14:13:56.761130 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:56.761098    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-95wtb"]
Jul 07 14:13:56.763872 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:13:56.763844    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0588c337_000a_4777_bfeb_67c419ebd711.slice/crio-b7037be24c5ccca4f9f1bddc74ee09bf15b5a9a880f5dcaca6efeddeea60dcd5 WatchSource:0}: Error finding container b7037be24c5ccca4f9f1bddc74ee09bf15b5a9a880f5dcaca6efeddeea60dcd5: Status 404 returned error can't find the container with id b7037be24c5ccca4f9f1bddc74ee09bf15b5a9a880f5dcaca6efeddeea60dcd5
Jul 07 14:13:57.043433 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:57.043405    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x" event={"ID":"8b785d9c-7605-4b26-b4d5-ef63167b72d6","Type":"ContainerStarted","Data":"074912ab28ac5d4cfed7f09fda7b8f2e5318e2f3c212b15983ee1f8d9921baac"}
Jul 07 14:13:57.044452 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:57.044435    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95wtb" event={"ID":"0588c337-000a-4777-bfeb-67c419ebd711","Type":"ContainerStarted","Data":"b7037be24c5ccca4f9f1bddc74ee09bf15b5a9a880f5dcaca6efeddeea60dcd5"}
Jul 07 14:13:57.063305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:57.063261    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht89x" podStartSLOduration=17.956428195 podStartE2EDuration="20.063248227s" totalImagesPullingTime="2.106820032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:13:57.062868184 +0000 UTC m=+291.351777282" watchObservedRunningTime="2026-07-07 14:13:57.063248227 +0000 UTC m=+291.352157325"
Jul 07 14:13:58.711486 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:58.711457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:58.711899 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:58.711519    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:58.712005 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:58.711984    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:58.713706 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:58.713685    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecc8e344-fafd-48aa-9d19-b5f168f0ec43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jtnbn\" (UID: \"ecc8e344-fafd-48aa-9d19-b5f168f0ec43\") " pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:58.797777 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:58.797753    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jtnbn"
Jul 07 14:13:58.927232 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:58.927129    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jtnbn"]
Jul 07 14:13:58.929267 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:13:58.929243    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc8e344_fafd_48aa_9d19_b5f168f0ec43.slice/crio-c43a8e4275e44342135c99db223814a5e90b71cf840d0dba9cbfe888ef695232 WatchSource:0}: Error finding container c43a8e4275e44342135c99db223814a5e90b71cf840d0dba9cbfe888ef695232: Status 404 returned error can't find the container with id c43a8e4275e44342135c99db223814a5e90b71cf840d0dba9cbfe888ef695232
Jul 07 14:13:59.052095 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:59.052032    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jtnbn" event={"ID":"ecc8e344-fafd-48aa-9d19-b5f168f0ec43","Type":"ContainerStarted","Data":"c43a8e4275e44342135c99db223814a5e90b71cf840d0dba9cbfe888ef695232"}
Jul 07 14:13:59.053561 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:59.053539    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95wtb" event={"ID":"0588c337-000a-4777-bfeb-67c419ebd711","Type":"ContainerStarted","Data":"f76d4a5ed3f102bf29914f5818c99447ae1111deaec4bc33a2dce6f41154b69f"}
Jul 07 14:13:59.053653 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:59.053574    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95wtb" event={"ID":"0588c337-000a-4777-bfeb-67c419ebd711","Type":"ContainerStarted","Data":"4ad21d71194955ab14fca4c6e04548c71f62fe3da28d1180dbf6ee0eddd60ea4"}
Jul 07 14:13:59.053726 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:59.053677    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-95wtb"
Jul 07 14:13:59.073223 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:13:59.073141    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-95wtb" podStartSLOduration=251.686021986 podStartE2EDuration="4m13.073126627s" totalImagesPullingTime="1.387104641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:13:59.072131831 +0000 UTC m=+293.361040948" watchObservedRunningTime="2026-07-07 14:13:59.073126627 +0000 UTC m=+293.362035719"
Jul 07 14:14:03.070315 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:03.070235    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jtnbn" event={"ID":"ecc8e344-fafd-48aa-9d19-b5f168f0ec43","Type":"ContainerStarted","Data":"77f7801851338611348797d6cc1e77968bd4fd179070d48eb49f6e4ceaacf621"}
Jul 07 14:14:04.076915 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:04.076876    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jtnbn" event={"ID":"ecc8e344-fafd-48aa-9d19-b5f168f0ec43","Type":"ContainerStarted","Data":"d17db000e9d742b74ee2246a4e047b9d18599e331f0e2b744c8472e44ba2fd2a"}
Jul 07 14:14:06.085109 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:06.085080    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jtnbn" event={"ID":"ecc8e344-fafd-48aa-9d19-b5f168f0ec43","Type":"ContainerStarted","Data":"c1c55daa770294117fdfb3ce8fe52a562446bfeab311babe58a3874d275b1f54"}
Jul 07 14:14:06.094123 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:06.094104    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:14:06.094787 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:06.094766    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:14:06.100978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:06.100961    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 14:14:06.112893 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:06.112860    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jtnbn" podStartSLOduration=17.1935025 podStartE2EDuration="24.112848791s" totalImagesPullingTime="6.919346291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:13:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:06.112821702 +0000 UTC m=+300.401730799" watchObservedRunningTime="2026-07-07 14:14:06.112848791 +0000 UTC m=+300.401757888"
Jul 07 14:14:09.059744 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.059711    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-95wtb"
Jul 07 14:14:09.238322 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:09.238287    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x8nq2" podUID="015c7f85-80c7-42cf-8c77-6f22642344c1"
Jul 07 14:14:09.617791 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.617759    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d9ff8d777-5fj9n"]
Jul 07 14:14:09.621883 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.621843    2570 patch_prober.go:28] interesting pod/image-registry-7d9ff8d777-5fj9n container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 14:14:09.621996 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.621899    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" podUID="e8190923-ba50-4711-92ea-fd515ca6b73b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 14:14:09.689638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.689611    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:09.690137 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.690119    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aec2ddc5-6572-4ebe-a4f8-a42897d65002-service-ca-bundle\") pod \"router-default-74ff99db5f-jzfph\" (UID: \"aec2ddc5-6572-4ebe-a4f8-a42897d65002\") " pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:09.958990 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.958932    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kc898\""
Jul 07 14:14:09.967761 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:09.967740    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:10.095006 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:10.094976    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74ff99db5f-jzfph"]
Jul 07 14:14:10.097407 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:10.097381    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec2ddc5_6572_4ebe_a4f8_a42897d65002.slice/crio-bf7de86c11f319c51274a03d8439ef904a2eb6ab4ec5d36aa92d6501e311f313 WatchSource:0}: Error finding container bf7de86c11f319c51274a03d8439ef904a2eb6ab4ec5d36aa92d6501e311f313: Status 404 returned error can't find the container with id bf7de86c11f319c51274a03d8439ef904a2eb6ab4ec5d36aa92d6501e311f313
Jul 07 14:14:11.102758 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.102715    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74ff99db5f-jzfph" event={"ID":"aec2ddc5-6572-4ebe-a4f8-a42897d65002","Type":"ContainerStarted","Data":"7cd2f94388cd9add68edb841ec127065a42acd22d607f80a71e0ae0aba78a9c4"}
Jul 07 14:14:11.103175 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.102776    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74ff99db5f-jzfph" event={"ID":"aec2ddc5-6572-4ebe-a4f8-a42897d65002","Type":"ContainerStarted","Data":"bf7de86c11f319c51274a03d8439ef904a2eb6ab4ec5d36aa92d6501e311f313"}
Jul 07 14:14:11.125420 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.124746    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74ff99db5f-jzfph" podStartSLOduration=34.124723633 podStartE2EDuration="34.124723633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:11.123807971 +0000 UTC m=+305.412717090" watchObservedRunningTime="2026-07-07 14:14:11.124723633 +0000 UTC m=+305.413632735"
Jul 07 14:14:11.672710 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.672677    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9977769d-dm2vx"]
Jul 07 14:14:11.676008 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.675989    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.678494 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.678470    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 14:14:11.678807 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.678788    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 14:14:11.678896 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.678841    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 14:14:11.679085 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.679068    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 14:14:11.679189 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.679117    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 14:14:11.679390 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.679371    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 14:14:11.679472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.679444    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 14:14:11.679472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.679455    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8njng\""
Jul 07 14:14:11.687398 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.687375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9977769d-dm2vx"]
Jul 07 14:14:11.806884 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.806857    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-serving-cert\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.806986 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.806889    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-service-ca\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.806986 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.806908    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-oauth-serving-cert\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.806986 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.806937    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-oauth-config\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.807091 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.807008    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-config\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.807091 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.807055    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm7cq\" (UniqueName: \"kubernetes.io/projected/ff258dce-5680-47c5-b58c-63a8eee17ce2-kube-api-access-xm7cq\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.908220 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908195    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm7cq\" (UniqueName: \"kubernetes.io/projected/ff258dce-5680-47c5-b58c-63a8eee17ce2-kube-api-access-xm7cq\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.908321 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908240    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-serving-cert\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.908321 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908261    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-service-ca\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.908321 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908279    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-oauth-serving-cert\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.908532 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908415    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-oauth-config\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.908532 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908468    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-config\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.909023 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.908993    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-oauth-serving-cert\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.909131 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.909082    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-service-ca\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.909233 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.909135    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-config\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.911012 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.910981    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-serving-cert\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.911012 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.911000    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-oauth-config\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.916867 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.916840    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm7cq\" (UniqueName: \"kubernetes.io/projected/ff258dce-5680-47c5-b58c-63a8eee17ce2-kube-api-access-xm7cq\") pod \"console-c9977769d-dm2vx\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") " pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:11.968564 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.968509    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:11.971916 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.971716    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:11.986294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:11.986273    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:12.106244 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:12.106215    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:12.107351 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:12.107329    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74ff99db5f-jzfph"
Jul 07 14:14:12.115226 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:12.115206    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9977769d-dm2vx"]
Jul 07 14:14:12.117306 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:12.117283    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff258dce_5680_47c5_b58c_63a8eee17ce2.slice/crio-0ee155cab0cc62dd50fc8a35a7fad2c64db2714b6ae64ebce07c8790cc80f520 WatchSource:0}: Error finding container 0ee155cab0cc62dd50fc8a35a7fad2c64db2714b6ae64ebce07c8790cc80f520: Status 404 returned error can't find the container with id 0ee155cab0cc62dd50fc8a35a7fad2c64db2714b6ae64ebce07c8790cc80f520
Jul 07 14:14:12.119104 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:12.119089    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:14:13.110587 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:13.110557    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9977769d-dm2vx" event={"ID":"ff258dce-5680-47c5-b58c-63a8eee17ce2","Type":"ContainerStarted","Data":"0ee155cab0cc62dd50fc8a35a7fad2c64db2714b6ae64ebce07c8790cc80f520"}
Jul 07 14:14:16.121978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:16.121939    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9977769d-dm2vx" event={"ID":"ff258dce-5680-47c5-b58c-63a8eee17ce2","Type":"ContainerStarted","Data":"847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f"}
Jul 07 14:14:16.142821 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:16.142777    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9977769d-dm2vx" podStartSLOduration=1.5630102890000002 podStartE2EDuration="5.142763329s" totalImagesPullingTime="3.57975304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:16.141069974 +0000 UTC m=+310.429979071" watchObservedRunningTime="2026-07-07 14:14:16.142763329 +0000 UTC m=+310.431672427"
Jul 07 14:14:19.623173 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:19.623121    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:14:21.236774 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.236739    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:14:21.325252 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.325228    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-st6dx"]
Jul 07 14:14:21.330080 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.330054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.335309 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.335291    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 14:14:21.336110 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.336070    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 14:14:21.336274 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.336118    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 14:14:21.336349 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.336287    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 14:14:21.336411 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.336374    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z45xp\""
Jul 07 14:14:21.336925 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.336656    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 14:14:21.336925 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.336865    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 14:14:21.370981 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.370957    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-wtmp\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371079 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.370994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-sys\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371079 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371022    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371079 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371049    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-root\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371079 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371070    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-textfile\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371275 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371093    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-accelerators-collector-config\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371275 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371138    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-tls\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371275 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371196    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wqpd\" (UniqueName: \"kubernetes.io/projected/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-kube-api-access-4wqpd\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.371275 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.371234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-metrics-client-ca\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472204 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472172    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472310 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472205    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-root\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472310 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-textfile\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472310 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472244    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-accelerators-collector-config\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472310 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472271    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-tls\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472310 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472278    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-root\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472310 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wqpd\" (UniqueName: \"kubernetes.io/projected/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-kube-api-access-4wqpd\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472591 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-metrics-client-ca\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472591 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472406    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-wtmp\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472591 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472432    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-sys\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472591 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472498    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-sys\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472591 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472546    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-textfile\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472813 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472675    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-wtmp\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472862 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472844    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-accelerators-collector-config\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.472924 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.472907    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-metrics-client-ca\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.474550 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.474530    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-tls\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.474611 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.474530    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.484761 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.484740    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wqpd\" (UniqueName: \"kubernetes.io/projected/c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52-kube-api-access-4wqpd\") pod \"node-exporter-st6dx\" (UID: \"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52\") " pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.640575 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.640551    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-st6dx"
Jul 07 14:14:21.652476 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:21.652456    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ae9ae2_c8f0_4173_a82d_fbf000ec8a52.slice/crio-89b292f4c4eb89c210a4624ede54616bd6eb96df7dffed0cbe9bd4a6383fbd1c WatchSource:0}: Error finding container 89b292f4c4eb89c210a4624ede54616bd6eb96df7dffed0cbe9bd4a6383fbd1c: Status 404 returned error can't find the container with id 89b292f4c4eb89c210a4624ede54616bd6eb96df7dffed0cbe9bd4a6383fbd1c
Jul 07 14:14:21.986957 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.986891    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:21.987065 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.986961    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:21.991631 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:21.991611    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:22.142563 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.142536    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6dx" event={"ID":"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52","Type":"ContainerStarted","Data":"89b292f4c4eb89c210a4624ede54616bd6eb96df7dffed0cbe9bd4a6383fbd1c"}
Jul 07 14:14:22.146125 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.146109    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:14:22.375878 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.375850    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 14:14:22.379327 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.379308    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.382090 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.381983    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 14:14:22.382219 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382169    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 14:14:22.382219 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382169    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 14:14:22.382541 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382478    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 14:14:22.382646 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382629    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 14:14:22.382858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382776    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 14:14:22.382858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382841    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 14:14:22.382858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382851    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 14:14:22.383066 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.382868    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-z9fw2\""
Jul 07 14:14:22.383066 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.383002    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 14:14:22.399659 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.399639    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 14:14:22.479814 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479788    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-web-config\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479879    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479903    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479946    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.479994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-config-volume\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.480031    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6adaa101-e7dc-47be-b875-94065c3b1069-config-out\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480183 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.480134    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gh7h\" (UniqueName: \"kubernetes.io/projected/6adaa101-e7dc-47be-b875-94065c3b1069-kube-api-access-8gh7h\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480536 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.480195    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6adaa101-e7dc-47be-b875-94065c3b1069-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480536 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.480222    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480536 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.480257    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.480536 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.480286    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.580832 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580806    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580840    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580855    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-web-config\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580872    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580933    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580964    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-config-volume\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581019 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.580984    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6adaa101-e7dc-47be-b875-94065c3b1069-config-out\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.581029    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gh7h\" (UniqueName: \"kubernetes.io/projected/6adaa101-e7dc-47be-b875-94065c3b1069-kube-api-access-8gh7h\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.581080    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6adaa101-e7dc-47be-b875-94065c3b1069-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.581104    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.581138    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.581185    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.581395 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:22.581308    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-trusted-ca-bundle podName:6adaa101-e7dc-47be-b875-94065c3b1069 nodeName:}" failed. No retries permitted until 2026-07-07 14:14:23.081288252 +0000 UTC m=+317.370197332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6adaa101-e7dc-47be-b875-94065c3b1069") : configmap references non-existent config key: ca-bundle.crt
Jul 07 14:14:22.581946 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:22.581874    2570 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 07 14:14:22.581946 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:22.581934    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-main-tls podName:6adaa101-e7dc-47be-b875-94065c3b1069 nodeName:}" failed. No retries permitted until 2026-07-07 14:14:23.081915682 +0000 UTC m=+317.370824760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6adaa101-e7dc-47be-b875-94065c3b1069") : secret "alertmanager-main-tls" not found
Jul 07 14:14:22.582243 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.582217    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.582442 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.582372    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.584505 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.584457    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6adaa101-e7dc-47be-b875-94065c3b1069-config-out\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.584764 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.584710    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.584865 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.584809    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.584865 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.584845    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.586617 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.585560    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-config-volume\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.592747 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.592259    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.592935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.592759    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6adaa101-e7dc-47be-b875-94065c3b1069-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.593100 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.592885    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-web-config\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:22.594959 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:22.594937    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gh7h\" (UniqueName: \"kubernetes.io/projected/6adaa101-e7dc-47be-b875-94065c3b1069-kube-api-access-8gh7h\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:23.084571 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.084513    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:23.084683 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.084583    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:23.085374 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.085348    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6adaa101-e7dc-47be-b875-94065c3b1069-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:23.086592 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.086574    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6adaa101-e7dc-47be-b875-94065c3b1069-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6adaa101-e7dc-47be-b875-94065c3b1069\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:23.147300 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.147272    2570 generic.go:356] "Generic (PLEG): container finished" podID="c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52" containerID="72ef84c0458965a8bc81df51544a1555d892fb000ff0fed4858918d83d273aa7" exitCode=0
Jul 07 14:14:23.147418 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.147391    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6dx" event={"ID":"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52","Type":"ContainerDied","Data":"72ef84c0458965a8bc81df51544a1555d892fb000ff0fed4858918d83d273aa7"}
Jul 07 14:14:23.291597 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.291574    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 14:14:23.433395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:23.433371    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 14:14:23.434958 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:23.434933    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adaa101_e7dc_47be_b875_94065c3b1069.slice/crio-9961a64cf409f7b145835b96b489331a8bd4a552af8d4c2a220718662b842f97 WatchSource:0}: Error finding container 9961a64cf409f7b145835b96b489331a8bd4a552af8d4c2a220718662b842f97: Status 404 returned error can't find the container with id 9961a64cf409f7b145835b96b489331a8bd4a552af8d4c2a220718662b842f97
Jul 07 14:14:24.152332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:24.152305    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6dx" event={"ID":"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52","Type":"ContainerStarted","Data":"7e244d93994f12399aba313e19c4182950b2e65f4f65541bd15fd389e7a0d43f"}
Jul 07 14:14:24.152476 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:24.152350    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-st6dx" event={"ID":"c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52","Type":"ContainerStarted","Data":"df73f038bde5fe4eb5dff295afbbe2f5e870a0391e0c78fa1fa2b4bd5cdff961"}
Jul 07 14:14:24.153422 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:24.153398    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"9961a64cf409f7b145835b96b489331a8bd4a552af8d4c2a220718662b842f97"}
Jul 07 14:14:24.180973 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:24.180935    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-st6dx" podStartSLOduration=2.028834542 podStartE2EDuration="3.180922848s" totalImagesPullingTime="1.090977119s" totalInitContainerRuntime="61.111187ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:14:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:24.179033269 +0000 UTC m=+318.467942366" watchObservedRunningTime="2026-07-07 14:14:24.180922848 +0000 UTC m=+318.469831945"
Jul 07 14:14:25.158912 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.158888    2570 generic.go:356] "Generic (PLEG): container finished" podID="6adaa101-e7dc-47be-b875-94065c3b1069" containerID="150d4c4f1b9fbb5a4d6367381947ecb615ef96f6aa5e19d082e12ea886edb66c" exitCode=0
Jul 07 14:14:25.159316 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.158979    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerDied","Data":"150d4c4f1b9fbb5a4d6367381947ecb615ef96f6aa5e19d082e12ea886edb66c"}
Jul 07 14:14:25.570759 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.570690    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b756d5bf9-mm9fx"]
Jul 07 14:14:25.573861 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.573841    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606373 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606347    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606468 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606402    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6p6r\" (UniqueName: \"kubernetes.io/projected/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-kube-api-access-m6p6r\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606516 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606462    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-metrics-server-audit-profiles\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606516 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606486    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-audit-log\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606516 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606505    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-secret-metrics-server-tls\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606615 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606526    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-secret-metrics-server-client-certs\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.606615 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.606547    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-client-ca-bundle\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.613900 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.613877    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 14:14:25.614001 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.613878    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-67d4h6gsftvfg\""
Jul 07 14:14:25.615051 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.615034    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 14:14:25.615125 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.615035    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 14:14:25.615125 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.615097    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 14:14:25.627840 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.627817    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4srzt\""
Jul 07 14:14:25.628231 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.628212    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b756d5bf9-mm9fx"]
Jul 07 14:14:25.706978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.706957    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-client-ca-bundle\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707089 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.706990    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707089 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707048    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6p6r\" (UniqueName: \"kubernetes.io/projected/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-kube-api-access-m6p6r\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707089 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-metrics-server-audit-profiles\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707089 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-audit-log\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707323 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707110    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-secret-metrics-server-tls\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707323 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707142    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-secret-metrics-server-client-certs\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707594 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707572    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-audit-log\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.707910 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.707887    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.708168 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.708131    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-metrics-server-audit-profiles\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.709464 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.709437    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-secret-metrics-server-client-certs\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.709646 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.709625    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-client-ca-bundle\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.709714 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.709634    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-secret-metrics-server-tls\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.717895 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.717873    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6p6r\" (UniqueName: \"kubernetes.io/projected/1d4d4b03-5766-440a-a2dc-e1121a59e5dd-kube-api-access-m6p6r\") pod \"metrics-server-b756d5bf9-mm9fx\" (UID: \"1d4d4b03-5766-440a-a2dc-e1121a59e5dd\") " pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:25.883503 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:25.883479    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:26.015901 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.015873    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b756d5bf9-mm9fx"]
Jul 07 14:14:26.019198 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:26.019171    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4d4b03_5766_440a_a2dc_e1121a59e5dd.slice/crio-d360402bc889f698bf13ba7cce085508c9f231f1bd0aaca6dd0b01ef314a66fb WatchSource:0}: Error finding container d360402bc889f698bf13ba7cce085508c9f231f1bd0aaca6dd0b01ef314a66fb: Status 404 returned error can't find the container with id d360402bc889f698bf13ba7cce085508c9f231f1bd0aaca6dd0b01ef314a66fb
Jul 07 14:14:26.070188 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.070162    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"]
Jul 07 14:14:26.074761 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.074743    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.077544 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.077520    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 14:14:26.077658 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.077597    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fv9kt\""
Jul 07 14:14:26.086636 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.086619    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"]
Jul 07 14:14:26.111039 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.111015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwxr6\" (UniqueName: \"kubernetes.io/projected/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-kube-api-access-vwxr6\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.111173 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.111121    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.167978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.167917    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx" event={"ID":"1d4d4b03-5766-440a-a2dc-e1121a59e5dd","Type":"ContainerStarted","Data":"d360402bc889f698bf13ba7cce085508c9f231f1bd0aaca6dd0b01ef314a66fb"}
Jul 07 14:14:26.211623 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.211599    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.211730 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.211711    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwxr6\" (UniqueName: \"kubernetes.io/projected/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-kube-api-access-vwxr6\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.211805 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:26.211747    2570 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 07 14:14:26.211841 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:26.211809    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-monitoring-plugin-cert podName:cd9e31bd-2f43-439a-9e71-ae4feeb3d007 nodeName:}" failed. No retries permitted until 2026-07-07 14:14:26.711790107 +0000 UTC m=+321.000699236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-bthnm" (UID: "cd9e31bd-2f43-439a-9e71-ae4feeb3d007") : secret "monitoring-plugin-cert" not found
Jul 07 14:14:26.223613 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.223593    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwxr6\" (UniqueName: \"kubernetes.io/projected/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-kube-api-access-vwxr6\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.715692 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.715654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.718464 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.718441    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd9e31bd-2f43-439a-9e71-ae4feeb3d007-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-bthnm\" (UID: \"cd9e31bd-2f43-439a-9e71-ae4feeb3d007\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:26.986278 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:26.986254    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:27.141215 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:27.141137    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"]
Jul 07 14:14:27.144344 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:27.144320    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9e31bd_2f43_439a_9e71_ae4feeb3d007.slice/crio-eacf62f07c3c0d72f259d20be0372020a4ce42aeaef353916ebc3e30e9589e8b WatchSource:0}: Error finding container eacf62f07c3c0d72f259d20be0372020a4ce42aeaef353916ebc3e30e9589e8b: Status 404 returned error can't find the container with id eacf62f07c3c0d72f259d20be0372020a4ce42aeaef353916ebc3e30e9589e8b
Jul 07 14:14:27.173530 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:27.173446    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm" event={"ID":"cd9e31bd-2f43-439a-9e71-ae4feeb3d007","Type":"ContainerStarted","Data":"eacf62f07c3c0d72f259d20be0372020a4ce42aeaef353916ebc3e30e9589e8b"}
Jul 07 14:14:27.177168 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:27.177106    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"7a14efe0887a27190e88a4cae6c8c8aa611904fdf329cc4cfcde41339249d20d"}
Jul 07 14:14:27.177268 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:27.177207    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"dcdc2e9d0db66b210bcc16b3e2a29f56ca07a4b535612069631b1dd2a88221b9"}
Jul 07 14:14:27.177268 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:27.177253    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"dd9e4b2859bb7ff498e4cf96eff70ba43af794b0793688c8db3e557cc1660226"}
Jul 07 14:14:27.177370 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:27.177293    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"cf2b5f67e3703c8b80fa154dc63f39e88c3cbdda437170c719ac66f25410800f"}
Jul 07 14:14:28.186047 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.186011    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"ede0057ac0a7214d1ebedaa9d47ea0f6d72aa3a28bdc9427000b91c501a31461"}
Jul 07 14:14:28.186530 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.186094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6adaa101-e7dc-47be-b875-94065c3b1069","Type":"ContainerStarted","Data":"2446ee93db918e8f6ceab6736b3d638562ef1338b6d723921e2169c53a6a0543"}
Jul 07 14:14:28.225109 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.225056    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.26316448 podStartE2EDuration="6.22503707s" totalImagesPullingTime="3.946123727s" totalInitContainerRuntime="15.748863ms" isStatefulPod=false podCreationTimestamp="2026-07-07 14:14:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:28.223494025 +0000 UTC m=+322.512403161" watchObservedRunningTime="2026-07-07 14:14:28.22503707 +0000 UTC m=+322.513946170"
Jul 07 14:14:28.709453 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.709413    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b89cc584f-5qwhb"]
Jul 07 14:14:28.714189 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.714144    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.721315 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.721294    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 14:14:28.725211 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.725179    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b89cc584f-5qwhb"]
Jul 07 14:14:28.733031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733007    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-oauth-serving-cert\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.733143 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733075    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-oauth-config\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.733224 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-trusted-ca-bundle\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.733224 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733198    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-console-config\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.733332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733233    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-serving-cert\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.733332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733253    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6bpk\" (UniqueName: \"kubernetes.io/projected/3278b29d-635b-48b1-a285-5facc7b78184-kube-api-access-x6bpk\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.733332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.733299    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-service-ca\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.833759 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833734    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-oauth-config\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.833858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833770    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-trusted-ca-bundle\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.833858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833789    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-console-config\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.833858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833812    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-serving-cert\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.833858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833838    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6bpk\" (UniqueName: \"kubernetes.io/projected/3278b29d-635b-48b1-a285-5facc7b78184-kube-api-access-x6bpk\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.834087 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833880    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-service-ca\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.834087 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.833930    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-oauth-serving-cert\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.834480 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.834436    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-console-config\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.834681 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.834658    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-trusted-ca-bundle\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.835189 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.835141    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-service-ca\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.835279 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.835237    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-oauth-serving-cert\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.836158 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.836125    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-serving-cert\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.836470 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.836451    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-oauth-config\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:28.842495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:28.842475    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6bpk\" (UniqueName: \"kubernetes.io/projected/3278b29d-635b-48b1-a285-5facc7b78184-kube-api-access-x6bpk\") pod \"console-7b89cc584f-5qwhb\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") " pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:29.026199 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.026124    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:29.152954 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.152930    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b89cc584f-5qwhb"]
Jul 07 14:14:29.155359 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:14:29.155331    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3278b29d_635b_48b1_a285_5facc7b78184.slice/crio-4abb6528da4f3d36e0b47b977a32fbd48f5d9682e71e03b403b6df9eae5af7c7 WatchSource:0}: Error finding container 4abb6528da4f3d36e0b47b977a32fbd48f5d9682e71e03b403b6df9eae5af7c7: Status 404 returned error can't find the container with id 4abb6528da4f3d36e0b47b977a32fbd48f5d9682e71e03b403b6df9eae5af7c7
Jul 07 14:14:29.192212 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.192189    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm" event={"ID":"cd9e31bd-2f43-439a-9e71-ae4feeb3d007","Type":"ContainerStarted","Data":"46dfb61852f1fc0619ad32c93b41ac3b4494196b8903c5e74d384168b184ac76"}
Jul 07 14:14:29.192561 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.192368    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:29.193544 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.193521    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b89cc584f-5qwhb" event={"ID":"3278b29d-635b-48b1-a285-5facc7b78184","Type":"ContainerStarted","Data":"4abb6528da4f3d36e0b47b977a32fbd48f5d9682e71e03b403b6df9eae5af7c7"}
Jul 07 14:14:29.195682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.195259    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx" event={"ID":"1d4d4b03-5766-440a-a2dc-e1121a59e5dd","Type":"ContainerStarted","Data":"45cf218407b9fca9bd200473f41506e22f6c5fdd9c45f7709c4b7c2a804689f4"}
Jul 07 14:14:29.198191 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.198174    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm"
Jul 07 14:14:29.213805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.213767    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-bthnm" podStartSLOduration=1.531732289 podStartE2EDuration="3.213754287s" totalImagesPullingTime="1.682021998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:29.212574264 +0000 UTC m=+323.501483364" watchObservedRunningTime="2026-07-07 14:14:29.213754287 +0000 UTC m=+323.502663384"
Jul 07 14:14:29.239413 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:29.239361    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx" podStartSLOduration=1.434927282 podStartE2EDuration="4.239341503s" totalImagesPullingTime="2.804414221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:29.237685893 +0000 UTC m=+323.526595007" watchObservedRunningTime="2026-07-07 14:14:29.239341503 +0000 UTC m=+323.528250603"
Jul 07 14:14:30.202463 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:30.202425    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b89cc584f-5qwhb" event={"ID":"3278b29d-635b-48b1-a285-5facc7b78184","Type":"ContainerStarted","Data":"dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8"}
Jul 07 14:14:30.238962 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:30.238911    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b89cc584f-5qwhb" podStartSLOduration=2.238898616 podStartE2EDuration="2.238898616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:14:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:14:30.227278749 +0000 UTC m=+324.516187848" watchObservedRunningTime="2026-07-07 14:14:30.238898616 +0000 UTC m=+324.527807785"
Jul 07 14:14:34.636213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.636165    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" podUID="e8190923-ba50-4711-92ea-fd515ca6b73b" containerName="registry" containerID="cri-o://4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d" gracePeriod=30
Jul 07 14:14:34.862988 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.862965    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:14:34.983703 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983641    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-trusted-ca\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983703 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983674    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983703 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983700    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-bound-sa-token\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983722    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8190923-ba50-4711-92ea-fd515ca6b73b-ca-trust-extracted\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983745    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvjx5\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-kube-api-access-fvjx5\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983778    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-image-registry-private-configuration\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983809    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-certificates\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.983935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.983835    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-installation-pull-secrets\") pod \"e8190923-ba50-4711-92ea-fd515ca6b73b\" (UID: \"e8190923-ba50-4711-92ea-fd515ca6b73b\") "
Jul 07 14:14:34.984196 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.984090    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:14:34.984378 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.984315    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:14:34.986303 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.986248    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:14:34.986303 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.986263    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:14:34.986454 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.986394    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:14:34.986454 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.986442    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-kube-api-access-fvjx5" (OuterVolumeSpecName: "kube-api-access-fvjx5") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "kube-api-access-fvjx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:14:34.986521 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.986444    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:14:34.992607 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:34.992584    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8190923-ba50-4711-92ea-fd515ca6b73b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e8190923-ba50-4711-92ea-fd515ca6b73b" (UID: "e8190923-ba50-4711-92ea-fd515ca6b73b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:14:35.085401 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085378    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-bound-sa-token\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085401 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085398    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e8190923-ba50-4711-92ea-fd515ca6b73b-ca-trust-extracted\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085520 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085409    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvjx5\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-kube-api-access-fvjx5\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085520 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085419    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-image-registry-private-configuration\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085520 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085429    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-certificates\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085520 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085437    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e8190923-ba50-4711-92ea-fd515ca6b73b-installation-pull-secrets\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085520 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085445    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8190923-ba50-4711-92ea-fd515ca6b73b-trusted-ca\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.085520 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.085453    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e8190923-ba50-4711-92ea-fd515ca6b73b-registry-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:14:35.221267 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.221244    2570 generic.go:356] "Generic (PLEG): container finished" podID="e8190923-ba50-4711-92ea-fd515ca6b73b" containerID="4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d" exitCode=0
Jul 07 14:14:35.221357 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.221294    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n"
Jul 07 14:14:35.221357 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.221318    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" event={"ID":"e8190923-ba50-4711-92ea-fd515ca6b73b","Type":"ContainerDied","Data":"4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d"}
Jul 07 14:14:35.221426 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.221359    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d9ff8d777-5fj9n" event={"ID":"e8190923-ba50-4711-92ea-fd515ca6b73b","Type":"ContainerDied","Data":"d4b351da8b1034cde350fbf0334d2956c76097326c7611a295caa8dad933b7fe"}
Jul 07 14:14:35.221426 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.221381    2570 scope.go:117] "RemoveContainer" containerID="4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d"
Jul 07 14:14:35.229471 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.229453    2570 scope.go:117] "RemoveContainer" containerID="4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d"
Jul 07 14:14:35.229725 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:14:35.229705    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d\": container with ID starting with 4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d not found: ID does not exist" containerID="4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d"
Jul 07 14:14:35.229793 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.229732    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d"} err="failed to get container status \"4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d\": rpc error: code = NotFound desc = could not find container \"4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d\": container with ID starting with 4c88d194a65e93d28e0b59b42a5f1e5c967e6473ff12b77fa62dd06eae036a3d not found: ID does not exist"
Jul 07 14:14:35.243371 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.243325    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d9ff8d777-5fj9n"]
Jul 07 14:14:35.246468 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:35.246448    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d9ff8d777-5fj9n"]
Jul 07 14:14:36.241588 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:36.241551    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8190923-ba50-4711-92ea-fd515ca6b73b" path="/var/lib/kubelet/pods/e8190923-ba50-4711-92ea-fd515ca6b73b/volumes"
Jul 07 14:14:39.026631 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:39.026602    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:39.026956 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:39.026657    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:39.031592 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:39.031572    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:39.240099 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:39.240069    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:14:39.297228 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:39.297163    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9977769d-dm2vx"]
Jul 07 14:14:45.884397 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:45.884363    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:14:45.884887 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:14:45.884413    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:15:04.324301 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.324187    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-c9977769d-dm2vx" podUID="ff258dce-5680-47c5-b58c-63a8eee17ce2" containerName="console" containerID="cri-o://847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f" gracePeriod=15
Jul 07 14:15:04.560292 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.560265    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:15:04.699085 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699058    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-config\") pod \"ff258dce-5680-47c5-b58c-63a8eee17ce2\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") "
Jul 07 14:15:04.699217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699096    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-serving-cert\") pod \"ff258dce-5680-47c5-b58c-63a8eee17ce2\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") "
Jul 07 14:15:04.699217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699137    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-service-ca\") pod \"ff258dce-5680-47c5-b58c-63a8eee17ce2\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") "
Jul 07 14:15:04.699217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699172    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm7cq\" (UniqueName: \"kubernetes.io/projected/ff258dce-5680-47c5-b58c-63a8eee17ce2-kube-api-access-xm7cq\") pod \"ff258dce-5680-47c5-b58c-63a8eee17ce2\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") "
Jul 07 14:15:04.699217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699196    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-oauth-serving-cert\") pod \"ff258dce-5680-47c5-b58c-63a8eee17ce2\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") "
Jul 07 14:15:04.699395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699221    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-oauth-config\") pod \"ff258dce-5680-47c5-b58c-63a8eee17ce2\" (UID: \"ff258dce-5680-47c5-b58c-63a8eee17ce2\") "
Jul 07 14:15:04.699530 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699501    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-config" (OuterVolumeSpecName: "console-config") pod "ff258dce-5680-47c5-b58c-63a8eee17ce2" (UID: "ff258dce-5680-47c5-b58c-63a8eee17ce2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:15:04.699599 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699517    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff258dce-5680-47c5-b58c-63a8eee17ce2" (UID: "ff258dce-5680-47c5-b58c-63a8eee17ce2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:15:04.699658 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.699599    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff258dce-5680-47c5-b58c-63a8eee17ce2" (UID: "ff258dce-5680-47c5-b58c-63a8eee17ce2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:15:04.701401 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.701371    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff258dce-5680-47c5-b58c-63a8eee17ce2" (UID: "ff258dce-5680-47c5-b58c-63a8eee17ce2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:15:04.701532 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.701513    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff258dce-5680-47c5-b58c-63a8eee17ce2-kube-api-access-xm7cq" (OuterVolumeSpecName: "kube-api-access-xm7cq") pod "ff258dce-5680-47c5-b58c-63a8eee17ce2" (UID: "ff258dce-5680-47c5-b58c-63a8eee17ce2"). InnerVolumeSpecName "kube-api-access-xm7cq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:15:04.701597 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.701570    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff258dce-5680-47c5-b58c-63a8eee17ce2" (UID: "ff258dce-5680-47c5-b58c-63a8eee17ce2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:15:04.800699 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.800678    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:15:04.800699 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.800699    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-serving-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:15:04.800823 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.800709    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-service-ca\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:15:04.800823 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.800717    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm7cq\" (UniqueName: \"kubernetes.io/projected/ff258dce-5680-47c5-b58c-63a8eee17ce2-kube-api-access-xm7cq\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:15:04.800823 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.800726    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff258dce-5680-47c5-b58c-63a8eee17ce2-oauth-serving-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:15:04.800823 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:04.800734    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff258dce-5680-47c5-b58c-63a8eee17ce2-console-oauth-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:15:05.337964 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.337936    2570 generic.go:356] "Generic (PLEG): container finished" podID="ff258dce-5680-47c5-b58c-63a8eee17ce2" containerID="847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f" exitCode=0
Jul 07 14:15:05.338313 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.337993    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9977769d-dm2vx" event={"ID":"ff258dce-5680-47c5-b58c-63a8eee17ce2","Type":"ContainerDied","Data":"847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f"}
Jul 07 14:15:05.338313 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.338028    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9977769d-dm2vx" event={"ID":"ff258dce-5680-47c5-b58c-63a8eee17ce2","Type":"ContainerDied","Data":"0ee155cab0cc62dd50fc8a35a7fad2c64db2714b6ae64ebce07c8790cc80f520"}
Jul 07 14:15:05.338313 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.338049    2570 scope.go:117] "RemoveContainer" containerID="847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f"
Jul 07 14:15:05.338313 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.337994    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9977769d-dm2vx"
Jul 07 14:15:05.346899 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.346869    2570 scope.go:117] "RemoveContainer" containerID="847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f"
Jul 07 14:15:05.347216 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:15:05.347183    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f\": container with ID starting with 847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f not found: ID does not exist" containerID="847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f"
Jul 07 14:15:05.347273 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.347222    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f"} err="failed to get container status \"847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f\": rpc error: code = NotFound desc = could not find container \"847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f\": container with ID starting with 847aafebf54a9a1ed04a5769770713b6e80fb5f69efdef46f608a33de3906f4f not found: ID does not exist"
Jul 07 14:15:05.359263 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.359242    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9977769d-dm2vx"]
Jul 07 14:15:05.363036 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.363015    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c9977769d-dm2vx"]
Jul 07 14:15:05.889638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.889615    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:15:05.893525 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:05.893504    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b756d5bf9-mm9fx"
Jul 07 14:15:06.242097 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:06.242042    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff258dce-5680-47c5-b58c-63a8eee17ce2" path="/var/lib/kubelet/pods/ff258dce-5680-47c5-b58c-63a8eee17ce2/volumes"
Jul 07 14:15:07.346567 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:07.346484    2570 generic.go:356] "Generic (PLEG): container finished" podID="4a36602f-4dd6-4ca8-aff4-646066eaf071" containerID="6c96827ba38cf29006e5078cf4c2a138777abba6be0f9ce8cb2118ac4970aa62" exitCode=0
Jul 07 14:15:07.346889 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:07.346561    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g" event={"ID":"4a36602f-4dd6-4ca8-aff4-646066eaf071","Type":"ContainerDied","Data":"6c96827ba38cf29006e5078cf4c2a138777abba6be0f9ce8cb2118ac4970aa62"}
Jul 07 14:15:07.346956 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:07.346897    2570 scope.go:117] "RemoveContainer" containerID="6c96827ba38cf29006e5078cf4c2a138777abba6be0f9ce8cb2118ac4970aa62"
Jul 07 14:15:08.277225 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:08.277201    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74ff99db5f-jzfph_aec2ddc5-6572-4ebe-a4f8-a42897d65002/router/0.log"
Jul 07 14:15:08.292839 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:08.292819    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vshns_b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5/serve-healthcheck-canary/0.log"
Jul 07 14:15:08.352036 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:08.352012    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-m7z4g" event={"ID":"4a36602f-4dd6-4ca8-aff4-646066eaf071","Type":"ContainerStarted","Data":"6c05f827507901129cb026d76017fb5c949ef04a1331fb67a2b871e0ac2322ec"}
Jul 07 14:15:12.367517 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:12.367488    2570 generic.go:356] "Generic (PLEG): container finished" podID="f6756644-141f-4d50-84a0-7a342e9b80e1" containerID="27e376a5b777bd823eac18bc8e875c17a1d629272568ee9dd883a4fc1cb6fbad" exitCode=0
Jul 07 14:15:12.367849 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:12.367560    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx" event={"ID":"f6756644-141f-4d50-84a0-7a342e9b80e1","Type":"ContainerDied","Data":"27e376a5b777bd823eac18bc8e875c17a1d629272568ee9dd883a4fc1cb6fbad"}
Jul 07 14:15:12.367897 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:12.367866    2570 scope.go:117] "RemoveContainer" containerID="27e376a5b777bd823eac18bc8e875c17a1d629272568ee9dd883a4fc1cb6fbad"
Jul 07 14:15:13.372896 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:13.372855    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-nkmpx" event={"ID":"f6756644-141f-4d50-84a0-7a342e9b80e1","Type":"ContainerStarted","Data":"7bcf80930311a82fcc08acbcbbafa8e94edeec5d3f9bed637740117551002112"}
Jul 07 14:15:19.118633 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:19.118583    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:15:19.120958 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:19.120932    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/015c7f85-80c7-42cf-8c77-6f22642344c1-metrics-certs\") pod \"network-metrics-daemon-x8nq2\" (UID: \"015c7f85-80c7-42cf-8c77-6f22642344c1\") " pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:15:19.140322 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:19.140300    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x2hv7\""
Jul 07 14:15:19.148723 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:19.148701    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8nq2"
Jul 07 14:15:19.284325 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:19.284289    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x8nq2"]
Jul 07 14:15:19.287217 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:15:19.287187    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015c7f85_80c7_42cf_8c77_6f22642344c1.slice/crio-5f77ad0a57caaa5e8b5f0199599dbbba761bb99dd6cc33f0fe68f5ee3db8282b WatchSource:0}: Error finding container 5f77ad0a57caaa5e8b5f0199599dbbba761bb99dd6cc33f0fe68f5ee3db8282b: Status 404 returned error can't find the container with id 5f77ad0a57caaa5e8b5f0199599dbbba761bb99dd6cc33f0fe68f5ee3db8282b
Jul 07 14:15:19.394745 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:19.394647    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x8nq2" event={"ID":"015c7f85-80c7-42cf-8c77-6f22642344c1","Type":"ContainerStarted","Data":"5f77ad0a57caaa5e8b5f0199599dbbba761bb99dd6cc33f0fe68f5ee3db8282b"}
Jul 07 14:15:21.409012 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:21.408019    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x8nq2" event={"ID":"015c7f85-80c7-42cf-8c77-6f22642344c1","Type":"ContainerStarted","Data":"064e28efe52f07c7468247748e88a0a81316d1104fe7edb0bc48210c9d7872db"}
Jul 07 14:15:21.409012 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:21.408094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x8nq2" event={"ID":"015c7f85-80c7-42cf-8c77-6f22642344c1","Type":"ContainerStarted","Data":"77c7735a335be71e6df9fc1a596c8bccf4ab1f3e9073c8797a38fe35f69c3843"}
Jul 07 14:15:21.436747 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:21.436694    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x8nq2" podStartSLOduration=373.819652597 podStartE2EDuration="6m15.436676432s" totalImagesPullingTime="1.617023835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:15:21.434874914 +0000 UTC m=+375.723784013" watchObservedRunningTime="2026-07-07 14:15:21.436676432 +0000 UTC m=+375.725585529"
Jul 07 14:15:45.912776 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.912743    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6467bfc974-4z42n"]
Jul 07 14:15:45.913213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.913012    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff258dce-5680-47c5-b58c-63a8eee17ce2" containerName="console"
Jul 07 14:15:45.913213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.913024    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff258dce-5680-47c5-b58c-63a8eee17ce2" containerName="console"
Jul 07 14:15:45.913213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.913038    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8190923-ba50-4711-92ea-fd515ca6b73b" containerName="registry"
Jul 07 14:15:45.913213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.913044    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8190923-ba50-4711-92ea-fd515ca6b73b" containerName="registry"
Jul 07 14:15:45.913213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.913107    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8190923-ba50-4711-92ea-fd515ca6b73b" containerName="registry"
Jul 07 14:15:45.913213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.913117    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff258dce-5680-47c5-b58c-63a8eee17ce2" containerName="console"
Jul 07 14:15:45.915224 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.915195    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:45.920884 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.920859    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 07 14:15:45.921295 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.921261    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zkk8q\""
Jul 07 14:15:45.921295 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.921277    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 07 14:15:45.921484 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.921399    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 07 14:15:45.921484 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.921425    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 07 14:15:45.922326 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.922025    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 07 14:15:45.926794 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.926770    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 07 14:15:45.946356 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:45.946334    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6467bfc974-4z42n"]
Jul 07 14:15:46.010319 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-secret-telemeter-client\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010410 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010330    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-telemeter-client-tls\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010410 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010352    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010410 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-federate-client-tls\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010410 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010400    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-metrics-client-ca\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010538 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5gzz\" (UniqueName: \"kubernetes.io/projected/393cd7a0-39cb-4673-907b-70a42b16368c-kube-api-access-d5gzz\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010538 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010448    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-serving-certs-ca-bundle\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.010538 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.010468    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.110948 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.110921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-telemeter-client-tls\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111034 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.110958    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111034 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.110976    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-federate-client-tls\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111034 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.110995    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-metrics-client-ca\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111142 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.111039    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5gzz\" (UniqueName: \"kubernetes.io/projected/393cd7a0-39cb-4673-907b-70a42b16368c-kube-api-access-d5gzz\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111142 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.111072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-serving-certs-ca-bundle\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111142 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.111096    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111438 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.111410    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-secret-telemeter-client\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.111947 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.111910    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-serving-certs-ca-bundle\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.112062 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.111990    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-metrics-client-ca\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.112765 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.112733    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/393cd7a0-39cb-4673-907b-70a42b16368c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.113868 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.113845    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-federate-client-tls\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.114121 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.114097    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.114182 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.114115    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-secret-telemeter-client\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.114274 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.114250    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/393cd7a0-39cb-4673-907b-70a42b16368c-telemeter-client-tls\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.121836 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.121809    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5gzz\" (UniqueName: \"kubernetes.io/projected/393cd7a0-39cb-4673-907b-70a42b16368c-kube-api-access-d5gzz\") pod \"telemeter-client-6467bfc974-4z42n\" (UID: \"393cd7a0-39cb-4673-907b-70a42b16368c\") " pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.224764 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.224711    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n"
Jul 07 14:15:46.362165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.362116    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6467bfc974-4z42n"]
Jul 07 14:15:46.370427 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:15:46.370403    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod393cd7a0_39cb_4673_907b_70a42b16368c.slice/crio-5d8fd519a3c95e76c773f320c7fe9fea05cdb287f236f8a8af48aba49627de40 WatchSource:0}: Error finding container 5d8fd519a3c95e76c773f320c7fe9fea05cdb287f236f8a8af48aba49627de40: Status 404 returned error can't find the container with id 5d8fd519a3c95e76c773f320c7fe9fea05cdb287f236f8a8af48aba49627de40
Jul 07 14:15:46.509939 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:46.509872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n" event={"ID":"393cd7a0-39cb-4673-907b-70a42b16368c","Type":"ContainerStarted","Data":"5d8fd519a3c95e76c773f320c7fe9fea05cdb287f236f8a8af48aba49627de40"}
Jul 07 14:15:49.525396 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:49.525358    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n" event={"ID":"393cd7a0-39cb-4673-907b-70a42b16368c","Type":"ContainerStarted","Data":"d738e46ca7a824dc8ca129e6245789df66243b246c10808c7d8cb9791d9a7022"}
Jul 07 14:15:49.525873 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:49.525428    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n" event={"ID":"393cd7a0-39cb-4673-907b-70a42b16368c","Type":"ContainerStarted","Data":"79ce9c6d4d8ef6fe017797d8fe2016d68c39d0a5b22eea1bef1744b79129fed1"}
Jul 07 14:15:49.525873 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:49.525463    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n" event={"ID":"393cd7a0-39cb-4673-907b-70a42b16368c","Type":"ContainerStarted","Data":"c37d5093fa39b9346b6367118d94168d6f76e15e6dbf784a6b11a3bfbb330f2e"}
Jul 07 14:15:49.552695 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:49.552648    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6467bfc974-4z42n" podStartSLOduration=2.288058048 podStartE2EDuration="4.552633299s" totalImagesPullingTime="2.264575251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:15:49.55228293 +0000 UTC m=+403.841192028" watchObservedRunningTime="2026-07-07 14:15:49.552633299 +0000 UTC m=+403.841542396"
Jul 07 14:15:50.264441 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.264414    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6696c875bc-qqrgz"]
Jul 07 14:15:50.266641 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.266624    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.290231 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.288474    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6696c875bc-qqrgz"]
Jul 07 14:15:50.344660 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344637    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-service-ca\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.344771 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344679    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzw7n\" (UniqueName: \"kubernetes.io/projected/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-kube-api-access-dzw7n\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.344771 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344745    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-oauth-serving-cert\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.344771 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344765    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-serving-cert\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.344903 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344784    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-trusted-ca-bundle\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.344903 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344809    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-oauth-config\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.344903 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.344823    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-config\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446069 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446042    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-service-ca\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446231 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446075    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzw7n\" (UniqueName: \"kubernetes.io/projected/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-kube-api-access-dzw7n\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446231 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446110    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-oauth-serving-cert\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446231 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446136    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-serving-cert\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446398 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-trusted-ca-bundle\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446398 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446374    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-oauth-config\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446398 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446394    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-config\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446947 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446916    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-service-ca\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.446947 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.446933    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-oauth-serving-cert\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.447123 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.447056    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-config\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.447263 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.447240    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-trusted-ca-bundle\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.448781 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.448760    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-serving-cert\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.449014 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.448989    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-oauth-config\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.455047 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.455013    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzw7n\" (UniqueName: \"kubernetes.io/projected/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-kube-api-access-dzw7n\") pod \"console-6696c875bc-qqrgz\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") " pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.575459 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.575400    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:15:50.719819 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:50.719742    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6696c875bc-qqrgz"]
Jul 07 14:15:50.723716 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:15:50.723684    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a5448f_ed68_40d5_976c_ab0e7ae095ef.slice/crio-682433fb652391931c96d16cc40da874ba1a8e806fa5c36480ae8148a7f643db WatchSource:0}: Error finding container 682433fb652391931c96d16cc40da874ba1a8e806fa5c36480ae8148a7f643db: Status 404 returned error can't find the container with id 682433fb652391931c96d16cc40da874ba1a8e806fa5c36480ae8148a7f643db
Jul 07 14:15:51.535499 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:51.535462    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696c875bc-qqrgz" event={"ID":"b5a5448f-ed68-40d5-976c-ab0e7ae095ef","Type":"ContainerStarted","Data":"bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4"}
Jul 07 14:15:51.535675 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:51.535520    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696c875bc-qqrgz" event={"ID":"b5a5448f-ed68-40d5-976c-ab0e7ae095ef","Type":"ContainerStarted","Data":"682433fb652391931c96d16cc40da874ba1a8e806fa5c36480ae8148a7f643db"}
Jul 07 14:15:51.555695 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:15:51.555642    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6696c875bc-qqrgz" podStartSLOduration=1.5556234230000001 podStartE2EDuration="1.555623423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:15:51.554133818 +0000 UTC m=+405.843042915" watchObservedRunningTime="2026-07-07 14:15:51.555623423 +0000 UTC m=+405.844532521"
Jul 07 14:16:00.576545 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:00.576518    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:16:00.576914 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:00.576566    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:16:00.580912 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:00.580889    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:16:01.582419 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.582392    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:16:01.641650 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.641609    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b89cc584f-5qwhb"]
Jul 07 14:16:01.728121 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.728091    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f67b4b668-l4kst"]
Jul 07 14:16:01.731384 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.731363    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.751369 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.751348    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f67b4b668-l4kst"]
Jul 07 14:16:01.826226 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826207    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-trusted-ca-bundle\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.826319 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826238    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-oauth-serving-cert\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.826319 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826264    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-serving-cert\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.826435 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2xr9\" (UniqueName: \"kubernetes.io/projected/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-kube-api-access-l2xr9\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.826435 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826374    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-service-ca\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.826507 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826431    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-config\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.826507 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.826474    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-oauth-config\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927216 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-config\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927316 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927231    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-oauth-config\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927316 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-trusted-ca-bundle\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927316 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927281    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-oauth-serving-cert\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927316 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927307    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-serving-cert\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927333    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2xr9\" (UniqueName: \"kubernetes.io/projected/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-kube-api-access-l2xr9\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.927495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.927352    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-service-ca\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.928077 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.928048    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-config\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.928077 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.928069    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-oauth-serving-cert\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.928272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.928139    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-trusted-ca-bundle\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.928272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.928204    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-service-ca\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.929949 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.929917    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-oauth-config\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.930037 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.929947    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-console-serving-cert\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:01.936579 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:01.936561    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2xr9\" (UniqueName: \"kubernetes.io/projected/d6c1bd6f-733a-4137-98e8-463fcc4cd7d9-kube-api-access-l2xr9\") pod \"console-6f67b4b668-l4kst\" (UID: \"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9\") " pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:02.040270 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:02.040246    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:02.167916 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:02.167891    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f67b4b668-l4kst"]
Jul 07 14:16:02.169949 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:16:02.169921    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c1bd6f_733a_4137_98e8_463fcc4cd7d9.slice/crio-e3d41b4099f62501c8ef0dc0d2b2ae5915f4e1d83fdde7d6f670b8cb6329c74b WatchSource:0}: Error finding container e3d41b4099f62501c8ef0dc0d2b2ae5915f4e1d83fdde7d6f670b8cb6329c74b: Status 404 returned error can't find the container with id e3d41b4099f62501c8ef0dc0d2b2ae5915f4e1d83fdde7d6f670b8cb6329c74b
Jul 07 14:16:02.580975 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:02.580872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f67b4b668-l4kst" event={"ID":"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9","Type":"ContainerStarted","Data":"6040bfe0c9983865fe9f5411e339847236f42dbbff2ac7789e67f12f79df1b7b"}
Jul 07 14:16:02.580975 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:02.580934    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f67b4b668-l4kst" event={"ID":"d6c1bd6f-733a-4137-98e8-463fcc4cd7d9","Type":"ContainerStarted","Data":"e3d41b4099f62501c8ef0dc0d2b2ae5915f4e1d83fdde7d6f670b8cb6329c74b"}
Jul 07 14:16:02.602592 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:02.602543    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f67b4b668-l4kst" podStartSLOduration=1.60252543 podStartE2EDuration="1.60252543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:16:02.601746663 +0000 UTC m=+416.890655761" watchObservedRunningTime="2026-07-07 14:16:02.60252543 +0000 UTC m=+416.891434527"
Jul 07 14:16:12.041034 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:12.040994    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:12.041488 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:12.041067    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:12.045898 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:12.045875    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:12.626894 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:12.626867    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f67b4b668-l4kst"
Jul 07 14:16:12.682390 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:12.682360    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6696c875bc-qqrgz"]
Jul 07 14:16:26.666442 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:26.666400    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b89cc584f-5qwhb" podUID="3278b29d-635b-48b1-a285-5facc7b78184" containerName="console" containerID="cri-o://dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8" gracePeriod=15
Jul 07 14:16:26.903103 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:26.903079    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:16:27.009980 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.009902    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-serving-cert\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.009980 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.009975    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-trusted-ca-bundle\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.010255 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010001    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-console-config\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.010255 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010076    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-service-ca\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.010255 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010163    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-oauth-serving-cert\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.010255 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010184    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-oauth-config\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.010255 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010207    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6bpk\" (UniqueName: \"kubernetes.io/projected/3278b29d-635b-48b1-a285-5facc7b78184-kube-api-access-x6bpk\") pod \"3278b29d-635b-48b1-a285-5facc7b78184\" (UID: \"3278b29d-635b-48b1-a285-5facc7b78184\") "
Jul 07 14:16:27.010498 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010411    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:27.010613 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010583    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:27.010688 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010632    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-console-config" (OuterVolumeSpecName: "console-config") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:27.010688 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.010670    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-service-ca" (OuterVolumeSpecName: "service-ca") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:27.011940 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.011921    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:16:27.012299 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.012279    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3278b29d-635b-48b1-a285-5facc7b78184-kube-api-access-x6bpk" (OuterVolumeSpecName: "kube-api-access-x6bpk") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "kube-api-access-x6bpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:16:27.012412 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.012384    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3278b29d-635b-48b1-a285-5facc7b78184" (UID: "3278b29d-635b-48b1-a285-5facc7b78184"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:16:27.111648 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111620    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-oauth-serving-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.111648 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111642    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-oauth-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.111648 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111651    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6bpk\" (UniqueName: \"kubernetes.io/projected/3278b29d-635b-48b1-a285-5facc7b78184-kube-api-access-x6bpk\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.111844 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111659    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3278b29d-635b-48b1-a285-5facc7b78184-console-serving-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.111844 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111668    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-trusted-ca-bundle\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.111844 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111676    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-console-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.111844 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.111683    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3278b29d-635b-48b1-a285-5facc7b78184-service-ca\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:27.687436 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.687406    2570 generic.go:356] "Generic (PLEG): container finished" podID="3278b29d-635b-48b1-a285-5facc7b78184" containerID="dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8" exitCode=0
Jul 07 14:16:27.687838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.687444    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b89cc584f-5qwhb" event={"ID":"3278b29d-635b-48b1-a285-5facc7b78184","Type":"ContainerDied","Data":"dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8"}
Jul 07 14:16:27.687838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.687473    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b89cc584f-5qwhb"
Jul 07 14:16:27.687838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.687490    2570 scope.go:117] "RemoveContainer" containerID="dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8"
Jul 07 14:16:27.687838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.687480    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b89cc584f-5qwhb" event={"ID":"3278b29d-635b-48b1-a285-5facc7b78184","Type":"ContainerDied","Data":"4abb6528da4f3d36e0b47b977a32fbd48f5d9682e71e03b403b6df9eae5af7c7"}
Jul 07 14:16:27.695708 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.695694    2570 scope.go:117] "RemoveContainer" containerID="dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8"
Jul 07 14:16:27.695945 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:16:27.695927    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8\": container with ID starting with dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8 not found: ID does not exist" containerID="dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8"
Jul 07 14:16:27.695994 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.695952    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8"} err="failed to get container status \"dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8\": rpc error: code = NotFound desc = could not find container \"dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8\": container with ID starting with dab7acba2f9dd4f1d1217729edb43d8bdd92117b9f10711063cd33c353ef12e8 not found: ID does not exist"
Jul 07 14:16:27.709830 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.709809    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b89cc584f-5qwhb"]
Jul 07 14:16:27.712281 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:27.712263    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b89cc584f-5qwhb"]
Jul 07 14:16:28.241386 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:28.241355    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3278b29d-635b-48b1-a285-5facc7b78184" path="/var/lib/kubelet/pods/3278b29d-635b-48b1-a285-5facc7b78184/volumes"
Jul 07 14:16:37.704122 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:37.704077    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6696c875bc-qqrgz" podUID="b5a5448f-ed68-40d5-976c-ab0e7ae095ef" containerName="console" containerID="cri-o://bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4" gracePeriod=15
Jul 07 14:16:37.946766 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:37.946743    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:16:38.100809 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100787    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-serving-cert\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.100935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100836    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-oauth-serving-cert\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.100935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100855    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-trusted-ca-bundle\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.100935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100882    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-oauth-config\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.100935 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100921    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-service-ca\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.101126 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100951    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-config\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.101126 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.100976    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzw7n\" (UniqueName: \"kubernetes.io/projected/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-kube-api-access-dzw7n\") pod \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\" (UID: \"b5a5448f-ed68-40d5-976c-ab0e7ae095ef\") "
Jul 07 14:16:38.101257 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.101208    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:38.101411 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.101382    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:38.101485 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.101429    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:38.101485 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.101439    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-config" (OuterVolumeSpecName: "console-config") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:16:38.103016 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.102991    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:16:38.103186 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.103141    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:16:38.103256 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.103180    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-kube-api-access-dzw7n" (OuterVolumeSpecName: "kube-api-access-dzw7n") pod "b5a5448f-ed68-40d5-976c-ab0e7ae095ef" (UID: "b5a5448f-ed68-40d5-976c-ab0e7ae095ef"). InnerVolumeSpecName "kube-api-access-dzw7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:16:38.201629 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201606    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-service-ca\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.201629 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201626    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.201750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201635    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzw7n\" (UniqueName: \"kubernetes.io/projected/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-kube-api-access-dzw7n\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.201750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201646    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-serving-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.201750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201655    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-oauth-serving-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.201750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201662    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-trusted-ca-bundle\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.201750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.201670    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5a5448f-ed68-40d5-976c-ab0e7ae095ef-console-oauth-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:16:38.733029 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.732992    2570 generic.go:356] "Generic (PLEG): container finished" podID="b5a5448f-ed68-40d5-976c-ab0e7ae095ef" containerID="bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4" exitCode=0
Jul 07 14:16:38.733405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.733057    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696c875bc-qqrgz"
Jul 07 14:16:38.733405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.733077    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696c875bc-qqrgz" event={"ID":"b5a5448f-ed68-40d5-976c-ab0e7ae095ef","Type":"ContainerDied","Data":"bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4"}
Jul 07 14:16:38.733405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.733124    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696c875bc-qqrgz" event={"ID":"b5a5448f-ed68-40d5-976c-ab0e7ae095ef","Type":"ContainerDied","Data":"682433fb652391931c96d16cc40da874ba1a8e806fa5c36480ae8148a7f643db"}
Jul 07 14:16:38.733405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.733178    2570 scope.go:117] "RemoveContainer" containerID="bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4"
Jul 07 14:16:38.741851 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.741830    2570 scope.go:117] "RemoveContainer" containerID="bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4"
Jul 07 14:16:38.742136 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:16:38.742115    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4\": container with ID starting with bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4 not found: ID does not exist" containerID="bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4"
Jul 07 14:16:38.742242 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.742143    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4"} err="failed to get container status \"bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4\": rpc error: code = NotFound desc = could not find container \"bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4\": container with ID starting with bcb65bfd03688ef9f970c71ffd01c31eb92535f43de1ed6586dc3860a80d47e4 not found: ID does not exist"
Jul 07 14:16:38.768562 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.768537    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6696c875bc-qqrgz"]
Jul 07 14:16:38.779462 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:38.779434    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6696c875bc-qqrgz"]
Jul 07 14:16:40.240934 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:16:40.240901    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a5448f-ed68-40d5-976c-ab0e7ae095ef" path="/var/lib/kubelet/pods/b5a5448f-ed68-40d5-976c-ab0e7ae095ef/volumes"
Jul 07 14:18:18.017797 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.017713    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"]
Jul 07 14:18:18.018272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.018008    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3278b29d-635b-48b1-a285-5facc7b78184" containerName="console"
Jul 07 14:18:18.018272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.018019    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3278b29d-635b-48b1-a285-5facc7b78184" containerName="console"
Jul 07 14:18:18.018272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.018029    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a5448f-ed68-40d5-976c-ab0e7ae095ef" containerName="console"
Jul 07 14:18:18.018272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.018035    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a5448f-ed68-40d5-976c-ab0e7ae095ef" containerName="console"
Jul 07 14:18:18.018272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.018088    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3278b29d-635b-48b1-a285-5facc7b78184" containerName="console"
Jul 07 14:18:18.018272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.018097    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a5448f-ed68-40d5-976c-ab0e7ae095ef" containerName="console"
Jul 07 14:18:18.021014 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.020994    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.024477 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.024434    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 14:18:18.024653 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.024611    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wbkzn\""
Jul 07 14:18:18.024729 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.024680    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 14:18:18.082673 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.082639    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"]
Jul 07 14:18:18.211780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.211755    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.211898 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.211788    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfbws\" (UniqueName: \"kubernetes.io/projected/3eb6ac80-5104-44c5-8ecb-dca443e0048e-kube-api-access-dfbws\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.211898 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.211819    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.312771 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.312699    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.312771 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.312740    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfbws\" (UniqueName: \"kubernetes.io/projected/3eb6ac80-5104-44c5-8ecb-dca443e0048e-kube-api-access-dfbws\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.312942 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.312788    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.313217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.313191    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.313296 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.313199    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.322060 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.322034    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfbws\" (UniqueName: \"kubernetes.io/projected/3eb6ac80-5104-44c5-8ecb-dca443e0048e-kube-api-access-dfbws\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.331496 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.331476    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:18.461680 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:18.461643    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"]
Jul 07 14:18:18.464248 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:18:18.464219    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb6ac80_5104_44c5_8ecb_dca443e0048e.slice/crio-e2b565603b729e05bfb2acd838a1bacf6c44f46d94c13c4db1c726b012e2ec47 WatchSource:0}: Error finding container e2b565603b729e05bfb2acd838a1bacf6c44f46d94c13c4db1c726b012e2ec47: Status 404 returned error can't find the container with id e2b565603b729e05bfb2acd838a1bacf6c44f46d94c13c4db1c726b012e2ec47
Jul 07 14:18:19.131015 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:19.130975    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj" event={"ID":"3eb6ac80-5104-44c5-8ecb-dca443e0048e","Type":"ContainerStarted","Data":"e2b565603b729e05bfb2acd838a1bacf6c44f46d94c13c4db1c726b012e2ec47"}
Jul 07 14:18:27.166658 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:27.166623    2570 generic.go:356] "Generic (PLEG): container finished" podID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerID="0e62d3910579e4d06fb4b4bbe23928dad3a2fbcdbcd904c59c0f26eb1d27189b" exitCode=0
Jul 07 14:18:27.167069 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:27.166712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj" event={"ID":"3eb6ac80-5104-44c5-8ecb-dca443e0048e","Type":"ContainerDied","Data":"0e62d3910579e4d06fb4b4bbe23928dad3a2fbcdbcd904c59c0f26eb1d27189b"}
Jul 07 14:18:29.178088 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:29.178062    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj" event={"ID":"3eb6ac80-5104-44c5-8ecb-dca443e0048e","Type":"ContainerStarted","Data":"864e79c26fd515c39b4f6e7cb12cb399c5887e925367c0fb50083970421780ba"}
Jul 07 14:18:30.184562 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:30.184525    2570 generic.go:356] "Generic (PLEG): container finished" podID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerID="864e79c26fd515c39b4f6e7cb12cb399c5887e925367c0fb50083970421780ba" exitCode=0
Jul 07 14:18:30.184970 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:30.184588    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj" event={"ID":"3eb6ac80-5104-44c5-8ecb-dca443e0048e","Type":"ContainerDied","Data":"864e79c26fd515c39b4f6e7cb12cb399c5887e925367c0fb50083970421780ba"}
Jul 07 14:18:39.224750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:39.224714    2570 generic.go:356] "Generic (PLEG): container finished" podID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerID="0824e14feee7b289a9b0a64d96a9bec8b40055cf947077e622da19d051cfd1a2" exitCode=0
Jul 07 14:18:39.225192 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:39.224815    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj" event={"ID":"3eb6ac80-5104-44c5-8ecb-dca443e0048e","Type":"ContainerDied","Data":"0824e14feee7b289a9b0a64d96a9bec8b40055cf947077e622da19d051cfd1a2"}
Jul 07 14:18:40.398207 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.398185    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:40.474965 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.474942    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-bundle\") pod \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") "
Jul 07 14:18:40.475089 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.474997    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfbws\" (UniqueName: \"kubernetes.io/projected/3eb6ac80-5104-44c5-8ecb-dca443e0048e-kube-api-access-dfbws\") pod \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") "
Jul 07 14:18:40.475089 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.475028    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-util\") pod \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\" (UID: \"3eb6ac80-5104-44c5-8ecb-dca443e0048e\") "
Jul 07 14:18:40.475583 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.475559    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-bundle" (OuterVolumeSpecName: "bundle") pod "3eb6ac80-5104-44c5-8ecb-dca443e0048e" (UID: "3eb6ac80-5104-44c5-8ecb-dca443e0048e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:18:40.476975 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.476947    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb6ac80-5104-44c5-8ecb-dca443e0048e-kube-api-access-dfbws" (OuterVolumeSpecName: "kube-api-access-dfbws") pod "3eb6ac80-5104-44c5-8ecb-dca443e0048e" (UID: "3eb6ac80-5104-44c5-8ecb-dca443e0048e"). InnerVolumeSpecName "kube-api-access-dfbws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:18:40.479212 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.479193    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-util" (OuterVolumeSpecName: "util") pod "3eb6ac80-5104-44c5-8ecb-dca443e0048e" (UID: "3eb6ac80-5104-44c5-8ecb-dca443e0048e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:18:40.575755 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.575733    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-util\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:18:40.575755 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.575752    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3eb6ac80-5104-44c5-8ecb-dca443e0048e-bundle\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:18:40.575878 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:40.575762    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfbws\" (UniqueName: \"kubernetes.io/projected/3eb6ac80-5104-44c5-8ecb-dca443e0048e-kube-api-access-dfbws\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:18:41.235655 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:41.235626    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj" event={"ID":"3eb6ac80-5104-44c5-8ecb-dca443e0048e","Type":"ContainerDied","Data":"e2b565603b729e05bfb2acd838a1bacf6c44f46d94c13c4db1c726b012e2ec47"}
Jul 07 14:18:41.235772 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:41.235666    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589hstcj"
Jul 07 14:18:41.235772 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:41.235675    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2b565603b729e05bfb2acd838a1bacf6c44f46d94c13c4db1c726b012e2ec47"
Jul 07 14:18:47.151165 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151124    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"]
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151467    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="pull"
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151480    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="pull"
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151489    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="extract"
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151496    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="extract"
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151505    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="util"
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151512    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="util"
Jul 07 14:18:47.151638 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.151573    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eb6ac80-5104-44c5-8ecb-dca443e0048e" containerName="extract"
Jul 07 14:18:47.153471 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.153449    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.156687 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.156651    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 07 14:18:47.156687 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.156676    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-2ff4r\""
Jul 07 14:18:47.156878 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.156773    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 07 14:18:47.156878 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.156774    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 07 14:18:47.185050 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.185027    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"]
Jul 07 14:18:47.223009 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.222985    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c5475c55-cd4e-4079-acf0-b64be3cad2b3-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb\" (UID: \"c5475c55-cd4e-4079-acf0-b64be3cad2b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.223109 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.223034    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4vgk\" (UniqueName: \"kubernetes.io/projected/c5475c55-cd4e-4079-acf0-b64be3cad2b3-kube-api-access-q4vgk\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb\" (UID: \"c5475c55-cd4e-4079-acf0-b64be3cad2b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.324024 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.324004    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c5475c55-cd4e-4079-acf0-b64be3cad2b3-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb\" (UID: \"c5475c55-cd4e-4079-acf0-b64be3cad2b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.324128 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.324077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4vgk\" (UniqueName: \"kubernetes.io/projected/c5475c55-cd4e-4079-acf0-b64be3cad2b3-kube-api-access-q4vgk\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb\" (UID: \"c5475c55-cd4e-4079-acf0-b64be3cad2b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.326446 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.326423    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/c5475c55-cd4e-4079-acf0-b64be3cad2b3-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb\" (UID: \"c5475c55-cd4e-4079-acf0-b64be3cad2b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.364213 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.364195    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4vgk\" (UniqueName: \"kubernetes.io/projected/c5475c55-cd4e-4079-acf0-b64be3cad2b3-kube-api-access-q4vgk\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb\" (UID: \"c5475c55-cd4e-4079-acf0-b64be3cad2b3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.463789 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.463741    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:47.605182 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:47.605131    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"]
Jul 07 14:18:47.608991 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:18:47.608963    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5475c55_cd4e_4079_acf0_b64be3cad2b3.slice/crio-219d6ab35a0175b3d4036eeb8374479748ea34dd743e505a2a63aa75930799c8 WatchSource:0}: Error finding container 219d6ab35a0175b3d4036eeb8374479748ea34dd743e505a2a63aa75930799c8: Status 404 returned error can't find the container with id 219d6ab35a0175b3d4036eeb8374479748ea34dd743e505a2a63aa75930799c8
Jul 07 14:18:48.265957 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:48.265924    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb" event={"ID":"c5475c55-cd4e-4079-acf0-b64be3cad2b3","Type":"ContainerStarted","Data":"219d6ab35a0175b3d4036eeb8374479748ea34dd743e505a2a63aa75930799c8"}
Jul 07 14:18:51.281852 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.281818    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb" event={"ID":"c5475c55-cd4e-4079-acf0-b64be3cad2b3","Type":"ContainerStarted","Data":"0521244e241d8f2506d115043e802d5ebcd0d0753e765ae1ff5999c9997d5070"}
Jul 07 14:18:51.282244 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.281977    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:18:51.331344 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.331293    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb" podStartSLOduration=1.6876139970000001 podStartE2EDuration="4.331274598s" totalImagesPullingTime="2.643660601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:18:51.325514724 +0000 UTC m=+585.614423824" watchObservedRunningTime="2026-07-07 14:18:51.331274598 +0000 UTC m=+585.620183698"
Jul 07 14:18:51.370645 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.370616    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-xvc6l"]
Jul 07 14:18:51.372676 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.372657    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.378048 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.378031    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 07 14:18:51.379964 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.379947    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-rpmtd\""
Jul 07 14:18:51.380212 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.380196    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 07 14:18:51.395788 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.395769    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-xvc6l"]
Jul 07 14:18:51.455689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.455669    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-certificates\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.455790 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.455702    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtvc8\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-kube-api-access-mtvc8\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.455790 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.455728    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-cabundle0\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.556387 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.556336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtvc8\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-kube-api-access-mtvc8\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.556387 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.556372    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-cabundle0\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.556506 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.556415    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-certificates\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.556506 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:51.556485    2570 secret.go:281] references non-existent secret key: ca.crt
Jul 07 14:18:51.556506 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:51.556497    2570 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 07 14:18:51.556506 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:51.556504    2570 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-xvc6l: references non-existent secret key: ca.crt
Jul 07 14:18:51.556655 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:51.556546    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-certificates podName:a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30 nodeName:}" failed. No retries permitted until 2026-07-07 14:18:52.056531665 +0000 UTC m=+586.345440754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-certificates") pod "keda-operator-68bff59c-xvc6l" (UID: "a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30") : references non-existent secret key: ca.crt
Jul 07 14:18:51.556962 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.556945    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-cabundle0\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:51.588411 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:51.588387    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtvc8\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-kube-api-access-mtvc8\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:52.059649 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.059620    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-certificates\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:52.061814 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.061791    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30-certificates\") pod \"keda-operator-68bff59c-xvc6l\" (UID: \"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30\") " pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:52.142831 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.142810    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"]
Jul 07 14:18:52.144773 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.144760    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.151631 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.151606    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 07 14:18:52.162746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.162728    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"]
Jul 07 14:18:52.261291 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.261261    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-certificates\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.261416 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.261318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a45eaffe-19d6-4109-a3c9-f681d1bb264f-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.261416 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.261395    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq7vb\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-kube-api-access-hq7vb\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.282214 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.282190    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:52.351495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.351462    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-q8c2b"]
Jul 07 14:18:52.354075 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.354059    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.357578 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.357562    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 07 14:18:52.362335 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.361752    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-certificates\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.362335 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.361787    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a45eaffe-19d6-4109-a3c9-f681d1bb264f-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.362335 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.361819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq7vb\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-kube-api-access-hq7vb\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.362335 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:52.362091    2570 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Jul 07 14:18:52.362335 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:52.362111    2570 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v: secret "keda-metrics-apiserver-certs" not found
Jul 07 14:18:52.362335 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:18:52.362172    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-certificates podName:a45eaffe-19d6-4109-a3c9-f681d1bb264f nodeName:}" failed. No retries permitted until 2026-07-07 14:18:52.862140613 +0000 UTC m=+587.151049690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-certificates") pod "keda-metrics-apiserver-559f75f947-gnq2v" (UID: "a45eaffe-19d6-4109-a3c9-f681d1bb264f") : secret "keda-metrics-apiserver-certs" not found
Jul 07 14:18:52.362949 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.362928    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a45eaffe-19d6-4109-a3c9-f681d1bb264f-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.382308 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.382286    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-q8c2b"]
Jul 07 14:18:52.383807 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.383778    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq7vb\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-kube-api-access-hq7vb\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.443718 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.443692    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-xvc6l"]
Jul 07 14:18:52.445042 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:18:52.445019    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5af4da1_5d3a_4ed4_b4b6_04586fcf6f30.slice/crio-1337a2b8f00096e049507361a74f3e7cbf553f7c088b0dde22cc29643eed8b13 WatchSource:0}: Error finding container 1337a2b8f00096e049507361a74f3e7cbf553f7c088b0dde22cc29643eed8b13: Status 404 returned error can't find the container with id 1337a2b8f00096e049507361a74f3e7cbf553f7c088b0dde22cc29643eed8b13
Jul 07 14:18:52.462634 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.462616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d2ba5fad-d1d0-4941-a6f1-91e4388980ee-certificates\") pod \"keda-admission-5fbd5c4644-q8c2b\" (UID: \"d2ba5fad-d1d0-4941-a6f1-91e4388980ee\") " pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.462728 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.462660    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4drh\" (UniqueName: \"kubernetes.io/projected/d2ba5fad-d1d0-4941-a6f1-91e4388980ee-kube-api-access-p4drh\") pod \"keda-admission-5fbd5c4644-q8c2b\" (UID: \"d2ba5fad-d1d0-4941-a6f1-91e4388980ee\") " pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.563237 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.563218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d2ba5fad-d1d0-4941-a6f1-91e4388980ee-certificates\") pod \"keda-admission-5fbd5c4644-q8c2b\" (UID: \"d2ba5fad-d1d0-4941-a6f1-91e4388980ee\") " pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.563330 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.563259    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4drh\" (UniqueName: \"kubernetes.io/projected/d2ba5fad-d1d0-4941-a6f1-91e4388980ee-kube-api-access-p4drh\") pod \"keda-admission-5fbd5c4644-q8c2b\" (UID: \"d2ba5fad-d1d0-4941-a6f1-91e4388980ee\") " pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.565461 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.565445    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/d2ba5fad-d1d0-4941-a6f1-91e4388980ee-certificates\") pod \"keda-admission-5fbd5c4644-q8c2b\" (UID: \"d2ba5fad-d1d0-4941-a6f1-91e4388980ee\") " pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.581685 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.581636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4drh\" (UniqueName: \"kubernetes.io/projected/d2ba5fad-d1d0-4941-a6f1-91e4388980ee-kube-api-access-p4drh\") pod \"keda-admission-5fbd5c4644-q8c2b\" (UID: \"d2ba5fad-d1d0-4941-a6f1-91e4388980ee\") " pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.666122 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.666103    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:52.808015 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.807952    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-q8c2b"]
Jul 07 14:18:52.865997 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.865970    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-certificates\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:52.868298 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:52.868279    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a45eaffe-19d6-4109-a3c9-f681d1bb264f-certificates\") pod \"keda-metrics-apiserver-559f75f947-gnq2v\" (UID: \"a45eaffe-19d6-4109-a3c9-f681d1bb264f\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:53.054791 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:53.054759    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:53.235234 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:53.235203    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"]
Jul 07 14:18:53.242187 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:18:53.242161    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda45eaffe_19d6_4109_a3c9_f681d1bb264f.slice/crio-b1243137376fe9a4df3a1dfade6ee526fdb97c5d94071b110fa15ccaac8d00e1 WatchSource:0}: Error finding container b1243137376fe9a4df3a1dfade6ee526fdb97c5d94071b110fa15ccaac8d00e1: Status 404 returned error can't find the container with id b1243137376fe9a4df3a1dfade6ee526fdb97c5d94071b110fa15ccaac8d00e1
Jul 07 14:18:53.290299 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:53.290272    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v" event={"ID":"a45eaffe-19d6-4109-a3c9-f681d1bb264f","Type":"ContainerStarted","Data":"b1243137376fe9a4df3a1dfade6ee526fdb97c5d94071b110fa15ccaac8d00e1"}
Jul 07 14:18:53.291252 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:53.291230    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-xvc6l" event={"ID":"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30","Type":"ContainerStarted","Data":"1337a2b8f00096e049507361a74f3e7cbf553f7c088b0dde22cc29643eed8b13"}
Jul 07 14:18:53.292307 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:53.292286    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b" event={"ID":"d2ba5fad-d1d0-4941-a6f1-91e4388980ee","Type":"ContainerStarted","Data":"1c6d2dee0d19c11094184b07de566aa4be4f7e518acf89951059030869e7a5c3"}
Jul 07 14:18:55.305420 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:55.305373    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b" event={"ID":"d2ba5fad-d1d0-4941-a6f1-91e4388980ee","Type":"ContainerStarted","Data":"8dc1136521060ab30443f09bb359b17e38bb9f93de2943ee13396e4987da758c"}
Jul 07 14:18:55.305838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:55.305561    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:18:55.333511 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:55.333467    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b" podStartSLOduration=1.647170024 podStartE2EDuration="3.333449577s" totalImagesPullingTime="1.686279553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:18:55.332823162 +0000 UTC m=+589.621732258" watchObservedRunningTime="2026-07-07 14:18:55.333449577 +0000 UTC m=+589.622358677"
Jul 07 14:18:57.316365 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:57.316324    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-xvc6l" event={"ID":"a5af4da1-5d3a-4ed4-b4b6-04586fcf6f30","Type":"ContainerStarted","Data":"801a33685c8c9483478ec16f2d235b375192f72eeaeb7c2165cd85f3151beb96"}
Jul 07 14:18:57.316823 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:57.316462    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:18:57.317966 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:57.317935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v" event={"ID":"a45eaffe-19d6-4109-a3c9-f681d1bb264f","Type":"ContainerStarted","Data":"cc4cbb508bf5a3dd12d7a52ddfab41e39dac2fd7993871f13c2f57de67a71f10"}
Jul 07 14:18:57.318080 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:57.318062    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:18:57.342867 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:57.342823    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-xvc6l" podStartSLOduration=2.218481253 podStartE2EDuration="6.342809312s" totalImagesPullingTime="4.124328059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:18:57.34089899 +0000 UTC m=+591.629808097" watchObservedRunningTime="2026-07-07 14:18:57.342809312 +0000 UTC m=+591.631718411"
Jul 07 14:18:57.367029 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:18:57.366988    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v" podStartSLOduration=2.040682071 podStartE2EDuration="5.36697591s" totalImagesPullingTime="3.326293839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:18:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:18:57.366120839 +0000 UTC m=+591.655029936" watchObservedRunningTime="2026-07-07 14:18:57.36697591 +0000 UTC m=+591.655885007"
Jul 07 14:19:06.124659 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:19:06.124633    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:19:06.125191 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:19:06.125173    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:19:08.325852 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:19:08.325820    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-gnq2v"
Jul 07 14:19:12.287870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:19:12.287834    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-4ccpb"
Jul 07 14:19:16.312990 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:19:16.312961    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-q8c2b"
Jul 07 14:19:18.324392 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:19:18.324360    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-xvc6l"
Jul 07 14:20:00.243362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.243329    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-5756x"]
Jul 07 14:20:00.245505 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.245488    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.249642 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.249619    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 07 14:20:00.249866 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.249853    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 07 14:20:00.250544 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.250528    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-dnzdq\""
Jul 07 14:20:00.250544 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.250534    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 07 14:20:00.265099 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.265075    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-5756x"]
Jul 07 14:20:00.271636 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.271615    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.271714 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.271682    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f6w8\" (UniqueName: \"kubernetes.io/projected/ad33c3c8-043d-48b8-b083-48c47fd85a2c-kube-api-access-4f6w8\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.372383 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.372359    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f6w8\" (UniqueName: \"kubernetes.io/projected/ad33c3c8-043d-48b8-b083-48c47fd85a2c-kube-api-access-4f6w8\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.372552 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.372418    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.372552 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:20:00.372543    2570 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 07 14:20:00.372662 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:20:00.372606    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert podName:ad33c3c8-043d-48b8-b083-48c47fd85a2c nodeName:}" failed. No retries permitted until 2026-07-07 14:20:00.872586519 +0000 UTC m=+655.161495619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert") pod "kserve-controller-manager-7598d7bb4-5756x" (UID: "ad33c3c8-043d-48b8-b083-48c47fd85a2c") : secret "kserve-webhook-server-cert" not found
Jul 07 14:20:00.385641 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.385610    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f6w8\" (UniqueName: \"kubernetes.io/projected/ad33c3c8-043d-48b8-b083-48c47fd85a2c-kube-api-access-4f6w8\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.876460 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.876427    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:00.878749 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:00.878728    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert\") pod \"kserve-controller-manager-7598d7bb4-5756x\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") " pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:01.156113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:01.156037    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:01.282065 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:01.282046    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-5756x"]
Jul 07 14:20:01.284285 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:20:01.284255    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad33c3c8_043d_48b8_b083_48c47fd85a2c.slice/crio-ea7164157d272e5902cd5a648cfb63f794bae2b5370df932a0650000d131bde3 WatchSource:0}: Error finding container ea7164157d272e5902cd5a648cfb63f794bae2b5370df932a0650000d131bde3: Status 404 returned error can't find the container with id ea7164157d272e5902cd5a648cfb63f794bae2b5370df932a0650000d131bde3
Jul 07 14:20:01.285457 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:01.285437    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:20:01.605455 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:01.605423    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7598d7bb4-5756x" event={"ID":"ad33c3c8-043d-48b8-b083-48c47fd85a2c","Type":"ContainerStarted","Data":"ea7164157d272e5902cd5a648cfb63f794bae2b5370df932a0650000d131bde3"}
Jul 07 14:20:05.627765 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:05.627729    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7598d7bb4-5756x" event={"ID":"ad33c3c8-043d-48b8-b083-48c47fd85a2c","Type":"ContainerStarted","Data":"de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c"}
Jul 07 14:20:05.628116 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:05.627859    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:05.650958 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:05.650910    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7598d7bb4-5756x" podStartSLOduration=2.244980731 podStartE2EDuration="5.650897773s" totalImagesPullingTime="3.405917042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:20:05.648299488 +0000 UTC m=+659.937208584" watchObservedRunningTime="2026-07-07 14:20:05.650897773 +0000 UTC m=+659.939806870"
Jul 07 14:20:36.637472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:36.637441    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:39.313766 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.313724    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-5756x"]
Jul 07 14:20:39.314294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.313982    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7598d7bb4-5756x" podUID="ad33c3c8-043d-48b8-b083-48c47fd85a2c" containerName="manager" containerID="cri-o://de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c" gracePeriod=10
Jul 07 14:20:39.352964 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.352937    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-wx9bg"]
Jul 07 14:20:39.354983 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.354967    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.371727 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.371702    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-wx9bg"]
Jul 07 14:20:39.471803 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.471764    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4zck\" (UniqueName: \"kubernetes.io/projected/b4821464-b1e8-4a05-b898-1be7fa34498d-kube-api-access-t4zck\") pod \"kserve-controller-manager-7598d7bb4-wx9bg\" (UID: \"b4821464-b1e8-4a05-b898-1be7fa34498d\") " pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.471934 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.471862    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4821464-b1e8-4a05-b898-1be7fa34498d-cert\") pod \"kserve-controller-manager-7598d7bb4-wx9bg\" (UID: \"b4821464-b1e8-4a05-b898-1be7fa34498d\") " pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.548215 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.548195    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:39.572754 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.572699    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4zck\" (UniqueName: \"kubernetes.io/projected/b4821464-b1e8-4a05-b898-1be7fa34498d-kube-api-access-t4zck\") pod \"kserve-controller-manager-7598d7bb4-wx9bg\" (UID: \"b4821464-b1e8-4a05-b898-1be7fa34498d\") " pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.572754 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.572744    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4821464-b1e8-4a05-b898-1be7fa34498d-cert\") pod \"kserve-controller-manager-7598d7bb4-wx9bg\" (UID: \"b4821464-b1e8-4a05-b898-1be7fa34498d\") " pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.574997 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.574973    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4821464-b1e8-4a05-b898-1be7fa34498d-cert\") pod \"kserve-controller-manager-7598d7bb4-wx9bg\" (UID: \"b4821464-b1e8-4a05-b898-1be7fa34498d\") " pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.587511 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.587487    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4zck\" (UniqueName: \"kubernetes.io/projected/b4821464-b1e8-4a05-b898-1be7fa34498d-kube-api-access-t4zck\") pod \"kserve-controller-manager-7598d7bb4-wx9bg\" (UID: \"b4821464-b1e8-4a05-b898-1be7fa34498d\") " pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.673743 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.673718    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f6w8\" (UniqueName: \"kubernetes.io/projected/ad33c3c8-043d-48b8-b083-48c47fd85a2c-kube-api-access-4f6w8\") pod \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") "
Jul 07 14:20:39.673844 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.673775    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert\") pod \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\" (UID: \"ad33c3c8-043d-48b8-b083-48c47fd85a2c\") "
Jul 07 14:20:39.675622 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.675595    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert" (OuterVolumeSpecName: "cert") pod "ad33c3c8-043d-48b8-b083-48c47fd85a2c" (UID: "ad33c3c8-043d-48b8-b083-48c47fd85a2c"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:20:39.675730 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.675711    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad33c3c8-043d-48b8-b083-48c47fd85a2c-kube-api-access-4f6w8" (OuterVolumeSpecName: "kube-api-access-4f6w8") pod "ad33c3c8-043d-48b8-b083-48c47fd85a2c" (UID: "ad33c3c8-043d-48b8-b083-48c47fd85a2c"). InnerVolumeSpecName "kube-api-access-4f6w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:20:39.686837 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.686820    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:39.774776 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.774748    2570 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad33c3c8-043d-48b8-b083-48c47fd85a2c-cert\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:20:39.774880 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.774781    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f6w8\" (UniqueName: \"kubernetes.io/projected/ad33c3c8-043d-48b8-b083-48c47fd85a2c-kube-api-access-4f6w8\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:20:39.787558 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.787524    2570 generic.go:356] "Generic (PLEG): container finished" podID="ad33c3c8-043d-48b8-b083-48c47fd85a2c" containerID="de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c" exitCode=0
Jul 07 14:20:39.787692 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.787580    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7598d7bb4-5756x"
Jul 07 14:20:39.787692 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.787613    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7598d7bb4-5756x" event={"ID":"ad33c3c8-043d-48b8-b083-48c47fd85a2c","Type":"ContainerDied","Data":"de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c"}
Jul 07 14:20:39.787808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.787693    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7598d7bb4-5756x" event={"ID":"ad33c3c8-043d-48b8-b083-48c47fd85a2c","Type":"ContainerDied","Data":"ea7164157d272e5902cd5a648cfb63f794bae2b5370df932a0650000d131bde3"}
Jul 07 14:20:39.787808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.787739    2570 scope.go:117] "RemoveContainer" containerID="de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c"
Jul 07 14:20:39.796394 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.796370    2570 scope.go:117] "RemoveContainer" containerID="de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c"
Jul 07 14:20:39.796682 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:20:39.796660    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c\": container with ID starting with de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c not found: ID does not exist" containerID="de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c"
Jul 07 14:20:39.796760 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.796690    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c"} err="failed to get container status \"de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c\": rpc error: code = NotFound desc = could not find container \"de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c\": container with ID starting with de96ccfc91d687b12987aa4538d77811a37c67d8fce66a2a25a013980e185f7c not found: ID does not exist"
Jul 07 14:20:39.811327 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.811293    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-wx9bg"]
Jul 07 14:20:39.813471 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:20:39.813450    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4821464_b1e8_4a05_b898_1be7fa34498d.slice/crio-5258c419fc2b5432c639f5c0b4566106d2118c7438df4ebdb15834833a040ddf WatchSource:0}: Error finding container 5258c419fc2b5432c639f5c0b4566106d2118c7438df4ebdb15834833a040ddf: Status 404 returned error can't find the container with id 5258c419fc2b5432c639f5c0b4566106d2118c7438df4ebdb15834833a040ddf
Jul 07 14:20:39.814251 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.814233    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-5756x"]
Jul 07 14:20:39.823355 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:39.823313    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7598d7bb4-5756x"]
Jul 07 14:20:40.242163 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:40.242127    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad33c3c8-043d-48b8-b083-48c47fd85a2c" path="/var/lib/kubelet/pods/ad33c3c8-043d-48b8-b083-48c47fd85a2c/volumes"
Jul 07 14:20:40.794617 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:40.794568    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg" event={"ID":"b4821464-b1e8-4a05-b898-1be7fa34498d","Type":"ContainerStarted","Data":"3d95d87f32ca8b9928b95336f19da83e8dfcc1ed34981ace29c139e2c0205c34"}
Jul 07 14:20:40.795025 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:40.794688    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:20:40.795025 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:20:40.794712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg" event={"ID":"b4821464-b1e8-4a05-b898-1be7fa34498d","Type":"ContainerStarted","Data":"5258c419fc2b5432c639f5c0b4566106d2118c7438df4ebdb15834833a040ddf"}
Jul 07 14:21:11.806382 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:11.806348    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg"
Jul 07 14:21:11.835851 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:11.835798    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7598d7bb4-wx9bg" podStartSLOduration=32.243583263 podStartE2EDuration="32.835782493s" totalImagesPullingTime="592.19923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:20:40.818398895 +0000 UTC m=+695.107308013" watchObservedRunningTime="2026-07-07 14:21:11.835782493 +0000 UTC m=+726.124691591"
Jul 07 14:21:13.559896 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.559863    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-nhzx2"]
Jul 07 14:21:13.560373 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.560358    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad33c3c8-043d-48b8-b083-48c47fd85a2c" containerName="manager"
Jul 07 14:21:13.560416 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.560376    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad33c3c8-043d-48b8-b083-48c47fd85a2c" containerName="manager"
Jul 07 14:21:13.560459 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.560448    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad33c3c8-043d-48b8-b083-48c47fd85a2c" containerName="manager"
Jul 07 14:21:13.564857 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.564839    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.574914 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.574897    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-8rdj7\""
Jul 07 14:21:13.577038 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.577025    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 07 14:21:13.617955 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.617935    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-nhzx2"]
Jul 07 14:21:13.620425 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.620403    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfs5v\" (UniqueName: \"kubernetes.io/projected/72155057-2313-4063-a9f4-2bc0aded0a36-kube-api-access-gfs5v\") pod \"model-serving-api-fd65d7d6-nhzx2\" (UID: \"72155057-2313-4063-a9f4-2bc0aded0a36\") " pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.620543 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.620443    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72155057-2313-4063-a9f4-2bc0aded0a36-tls-certs\") pod \"model-serving-api-fd65d7d6-nhzx2\" (UID: \"72155057-2313-4063-a9f4-2bc0aded0a36\") " pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.721087 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.721063    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72155057-2313-4063-a9f4-2bc0aded0a36-tls-certs\") pod \"model-serving-api-fd65d7d6-nhzx2\" (UID: \"72155057-2313-4063-a9f4-2bc0aded0a36\") " pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.721206 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.721142    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfs5v\" (UniqueName: \"kubernetes.io/projected/72155057-2313-4063-a9f4-2bc0aded0a36-kube-api-access-gfs5v\") pod \"model-serving-api-fd65d7d6-nhzx2\" (UID: \"72155057-2313-4063-a9f4-2bc0aded0a36\") " pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.723191 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.723173    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72155057-2313-4063-a9f4-2bc0aded0a36-tls-certs\") pod \"model-serving-api-fd65d7d6-nhzx2\" (UID: \"72155057-2313-4063-a9f4-2bc0aded0a36\") " pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.735483 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.735465    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfs5v\" (UniqueName: \"kubernetes.io/projected/72155057-2313-4063-a9f4-2bc0aded0a36-kube-api-access-gfs5v\") pod \"model-serving-api-fd65d7d6-nhzx2\" (UID: \"72155057-2313-4063-a9f4-2bc0aded0a36\") " pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:13.874469 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:13.874443    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:14.027044 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:14.027022    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-nhzx2"]
Jul 07 14:21:14.029298 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:21:14.029274    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72155057_2313_4063_a9f4_2bc0aded0a36.slice/crio-628ce27ebbfb38e4b0b88438f03bb5a203e45c5c8385f14de6924ab8e50542e6 WatchSource:0}: Error finding container 628ce27ebbfb38e4b0b88438f03bb5a203e45c5c8385f14de6924ab8e50542e6: Status 404 returned error can't find the container with id 628ce27ebbfb38e4b0b88438f03bb5a203e45c5c8385f14de6924ab8e50542e6
Jul 07 14:21:14.951922 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:14.951886    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-nhzx2" event={"ID":"72155057-2313-4063-a9f4-2bc0aded0a36","Type":"ContainerStarted","Data":"628ce27ebbfb38e4b0b88438f03bb5a203e45c5c8385f14de6924ab8e50542e6"}
Jul 07 14:21:16.961915 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:16.961880    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-nhzx2" event={"ID":"72155057-2313-4063-a9f4-2bc0aded0a36","Type":"ContainerStarted","Data":"5114895382c23025aa0f3c5307cc680b5183588dc506756aeddc36f5548a5bc4"}
Jul 07 14:21:16.962331 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:16.962081    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:21:16.999565 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:16.999523    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-nhzx2" podStartSLOduration=2.104806555 podStartE2EDuration="3.99951026s" totalImagesPullingTime="1.894703705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:21:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:21:16.996368784 +0000 UTC m=+731.285277883" watchObservedRunningTime="2026-07-07 14:21:16.99951026 +0000 UTC m=+731.288419357"
Jul 07 14:21:27.971143 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:21:27.971109    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-nhzx2"
Jul 07 14:23:01.004997 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.004962    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"]
Jul 07 14:23:01.007275 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.007255    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.009670 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.009629    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a2dea-kube-rbac-proxy-sar-config\""
Jul 07 14:23:01.009799 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.009784    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 07 14:23:01.010595 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.010575    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a2dea-predictor-serving-cert\""
Jul 07 14:23:01.010720 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.010635    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 07 14:23:01.011211 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.011190    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-j545g\""
Jul 07 14:23:01.018978 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.018950    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"]
Jul 07 14:23:01.110981 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.110950    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.111118 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.111000    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjb52\" (UniqueName: \"kubernetes.io/projected/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-api-access-fjb52\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.111118 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.111096    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-proxy-tls\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.212162 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.212112    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.212312 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.212180    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjb52\" (UniqueName: \"kubernetes.io/projected/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-api-access-fjb52\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.212312 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.212211    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-proxy-tls\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.212929 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.212903    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.214528 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.214498    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-proxy-tls\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.222524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.222502    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjb52\" (UniqueName: \"kubernetes.io/projected/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-api-access-fjb52\") pod \"error-404-isvc-a2dea-predictor-55844fbd46-hx9m7\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") " pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.321215 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.321123    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:01.466672 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:01.466649    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"]
Jul 07 14:23:01.468402 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:23:01.468362    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e9c248_50c3_4c8a_98ca_0f23ca9df0f9.slice/crio-38353c3d260c96a71d1235a1849b3d1e9fcd916283b08b43fc4fdcaeee3829af WatchSource:0}: Error finding container 38353c3d260c96a71d1235a1849b3d1e9fcd916283b08b43fc4fdcaeee3829af: Status 404 returned error can't find the container with id 38353c3d260c96a71d1235a1849b3d1e9fcd916283b08b43fc4fdcaeee3829af
Jul 07 14:23:02.443363 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:02.443264    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" event={"ID":"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9","Type":"ContainerStarted","Data":"38353c3d260c96a71d1235a1849b3d1e9fcd916283b08b43fc4fdcaeee3829af"}
Jul 07 14:23:14.512020 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:14.511933    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" event={"ID":"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9","Type":"ContainerStarted","Data":"b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28"}
Jul 07 14:23:16.525478 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:16.525444    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" event={"ID":"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9","Type":"ContainerStarted","Data":"4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34"}
Jul 07 14:23:16.525786 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:16.525505    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:16.557329 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:16.557283    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podStartSLOduration=1.74592419 podStartE2EDuration="16.557268433s" totalImagesPullingTime="14.811344243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:23:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:23:16.55666842 +0000 UTC m=+850.845577535" watchObservedRunningTime="2026-07-07 14:23:16.557268433 +0000 UTC m=+850.846177553"
Jul 07 14:23:17.530171 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:17.530123    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:17.531407 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:17.531376    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:23:18.534404 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:18.534363    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:23:23.538789 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:23.538758    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:23:23.539300 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:23.539268    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:23:33.539762 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:33.539702    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:23:43.539495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:43.539455    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:23:53.539362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:23:53.539316    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:24:03.540217 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:03.540110    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:24:06.154025 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:06.153998    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:24:06.155846 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:06.155824    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:24:30.922332 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:30.922304    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"]
Jul 07 14:24:30.922712 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:30.922513    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" containerID="cri-o://b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28" gracePeriod=30
Jul 07 14:24:30.922712 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:30.922544    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kube-rbac-proxy" containerID="cri-o://4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34" gracePeriod=30
Jul 07 14:24:31.118857 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.118826    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"]
Jul 07 14:24:31.121260 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.121232    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.123691 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.123671    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-520df-kube-rbac-proxy-sar-config\""
Jul 07 14:24:31.123791 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.123674    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-520df-predictor-serving-cert\""
Jul 07 14:24:31.134107 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.134088    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"]
Jul 07 14:24:31.194412 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.194347    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8sf\" (UniqueName: \"kubernetes.io/projected/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-api-access-xq8sf\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.194412 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.194377    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.194412 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.194401    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b481e4c-1994-4977-a83d-aec0e20ab4ca-proxy-tls\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.295511 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.295478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8sf\" (UniqueName: \"kubernetes.io/projected/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-api-access-xq8sf\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.295511 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.295511    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.295682 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.295535    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b481e4c-1994-4977-a83d-aec0e20ab4ca-proxy-tls\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.296140 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.296121    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.297940 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.297917    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b481e4c-1994-4977-a83d-aec0e20ab4ca-proxy-tls\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.310488 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.310470    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8sf\" (UniqueName: \"kubernetes.io/projected/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-api-access-xq8sf\") pod \"error-404-isvc-520df-predictor-696bbdd946-ns2nv\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") " pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.431309 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.431287    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.560606 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.560571    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"]
Jul 07 14:24:31.564260 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:24:31.564234    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b481e4c_1994_4977_a83d_aec0e20ab4ca.slice/crio-4416b413ff97ec54a3108afaa329054205a5c935ffce97dd1bad4c24cd74e69b WatchSource:0}: Error finding container 4416b413ff97ec54a3108afaa329054205a5c935ffce97dd1bad4c24cd74e69b: Status 404 returned error can't find the container with id 4416b413ff97ec54a3108afaa329054205a5c935ffce97dd1bad4c24cd74e69b
Jul 07 14:24:31.879580 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.879539    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" event={"ID":"3b481e4c-1994-4977-a83d-aec0e20ab4ca","Type":"ContainerStarted","Data":"de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120"}
Jul 07 14:24:31.879805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.879614    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" event={"ID":"3b481e4c-1994-4977-a83d-aec0e20ab4ca","Type":"ContainerStarted","Data":"6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649"}
Jul 07 14:24:31.879805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.879649    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:31.879805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.879677    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" event={"ID":"3b481e4c-1994-4977-a83d-aec0e20ab4ca","Type":"ContainerStarted","Data":"4416b413ff97ec54a3108afaa329054205a5c935ffce97dd1bad4c24cd74e69b"}
Jul 07 14:24:31.881728 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.881701    2570 generic.go:356] "Generic (PLEG): container finished" podID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerID="4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34" exitCode=2
Jul 07 14:24:31.881832 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.881746    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" event={"ID":"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9","Type":"ContainerDied","Data":"4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34"}
Jul 07 14:24:31.902035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:31.901996    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podStartSLOduration=0.901982167 podStartE2EDuration="901.982167ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:24:31.900789525 +0000 UTC m=+926.189698625" watchObservedRunningTime="2026-07-07 14:24:31.901982167 +0000 UTC m=+926.190891264"
Jul 07 14:24:32.887185 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:32.887132    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:32.888405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:32.888376    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 07 14:24:33.535104 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.535060    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.32:8643/healthz\": dial tcp 10.132.0.32:8643: connect: connection refused"
Jul 07 14:24:33.539372 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.539345    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Jul 07 14:24:33.771451 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.771427    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:24:33.815355 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.815296    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjb52\" (UniqueName: \"kubernetes.io/projected/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-api-access-fjb52\") pod \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") "
Jul 07 14:24:33.815445 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.815402    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-proxy-tls\") pod \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") "
Jul 07 14:24:33.815445 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.815439    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-rbac-proxy-sar-config\") pod \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\" (UID: \"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9\") "
Jul 07 14:24:33.815843 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.815820    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" (UID: "08e9c248-50c3-4c8a-98ca-0f23ca9df0f9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:24:33.817379 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.817344    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-api-access-fjb52" (OuterVolumeSpecName: "kube-api-access-fjb52") pod "08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" (UID: "08e9c248-50c3-4c8a-98ca-0f23ca9df0f9"). InnerVolumeSpecName "kube-api-access-fjb52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:24:33.817461 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.817442    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" (UID: "08e9c248-50c3-4c8a-98ca-0f23ca9df0f9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:24:33.892388 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.892366    2570 generic.go:356] "Generic (PLEG): container finished" podID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerID="b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28" exitCode=0
Jul 07 14:24:33.892689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.892411    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"
Jul 07 14:24:33.892689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.892442    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" event={"ID":"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9","Type":"ContainerDied","Data":"b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28"}
Jul 07 14:24:33.892689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.892519    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7" event={"ID":"08e9c248-50c3-4c8a-98ca-0f23ca9df0f9","Type":"ContainerDied","Data":"38353c3d260c96a71d1235a1849b3d1e9fcd916283b08b43fc4fdcaeee3829af"}
Jul 07 14:24:33.892689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.892541    2570 scope.go:117] "RemoveContainer" containerID="4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34"
Jul 07 14:24:33.892997 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.892971    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 07 14:24:33.901243 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.901228    2570 scope.go:117] "RemoveContainer" containerID="b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28"
Jul 07 14:24:33.908016 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.907996    2570 scope.go:117] "RemoveContainer" containerID="4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34"
Jul 07 14:24:33.908271 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:24:33.908253    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34\": container with ID starting with 4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34 not found: ID does not exist" containerID="4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34"
Jul 07 14:24:33.908335 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.908277    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34"} err="failed to get container status \"4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34\": rpc error: code = NotFound desc = could not find container \"4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34\": container with ID starting with 4d185089805adf2f8b5afe52cc6e2f4ec74fe3389ced4b45c0f8bc062deb3d34 not found: ID does not exist"
Jul 07 14:24:33.908335 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.908291    2570 scope.go:117] "RemoveContainer" containerID="b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28"
Jul 07 14:24:33.908540 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:24:33.908524    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28\": container with ID starting with b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28 not found: ID does not exist" containerID="b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28"
Jul 07 14:24:33.908585 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.908545    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28"} err="failed to get container status \"b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28\": rpc error: code = NotFound desc = could not find container \"b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28\": container with ID starting with b6fb394b231a80fa6219661f73cd28e808d21cdea0ebabfd641790be55829e28 not found: ID does not exist"
Jul 07 14:24:33.913614 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.913594    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"]
Jul 07 14:24:33.916189 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.916172    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:24:33.916189 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.916191    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:24:33.916296 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.916200    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjb52\" (UniqueName: \"kubernetes.io/projected/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9-kube-api-access-fjb52\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:24:33.917533 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:33.917516    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a2dea-predictor-55844fbd46-hx9m7"]
Jul 07 14:24:34.242031 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:34.242004    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" path="/var/lib/kubelet/pods/08e9c248-50c3-4c8a-98ca-0f23ca9df0f9/volumes"
Jul 07 14:24:38.897693 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:38.897666    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:24:38.898289 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:38.898264    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 07 14:24:48.898788 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:48.898746    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 07 14:24:58.899083 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:24:58.899048    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 07 14:25:08.899229 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:08.899192    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused"
Jul 07 14:25:18.900035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:18.900007    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:25:31.277433 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.277404    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"]
Jul 07 14:25:31.277898 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.277879    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container"
Jul 07 14:25:31.277967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.277902    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container"
Jul 07 14:25:31.277967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.277918    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kube-rbac-proxy"
Jul 07 14:25:31.277967 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.277926    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kube-rbac-proxy"
Jul 07 14:25:31.278110 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.278038    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kube-rbac-proxy"
Jul 07 14:25:31.278110 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.278050    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e9c248-50c3-4c8a-98ca-0f23ca9df0f9" containerName="kserve-container"
Jul 07 14:25:31.280805 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.280782    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.285584 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.285564    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ab362-predictor-serving-cert\""
Jul 07 14:25:31.285686 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.285591    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ab362-kube-rbac-proxy-sar-config\""
Jul 07 14:25:31.304745 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.304725    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"]
Jul 07 14:25:31.346423 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.346376    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.346542 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.346475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dda11bab-431b-4acd-b3c1-bc0b57459f9a-proxy-tls\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.346617 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.346544    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq8pw\" (UniqueName: \"kubernetes.io/projected/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-api-access-qq8pw\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.447533 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.447509    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.447657 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.447560    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dda11bab-431b-4acd-b3c1-bc0b57459f9a-proxy-tls\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.447657 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.447609    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq8pw\" (UniqueName: \"kubernetes.io/projected/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-api-access-qq8pw\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.448232 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.448204    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.450126 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.450097    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dda11bab-431b-4acd-b3c1-bc0b57459f9a-proxy-tls\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.456540 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.456516    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq8pw\" (UniqueName: \"kubernetes.io/projected/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-api-access-qq8pw\") pod \"error-404-isvc-ab362-predictor-56447f4fd8-2k4qr\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") " pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.592191 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.592122    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:31.728749 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:31.728725    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"]
Jul 07 14:25:31.731095 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:25:31.731068    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda11bab_431b_4acd_b3c1_bc0b57459f9a.slice/crio-dd7baac25a0f4b4ff95ae3aa62ee8eeb8d22b675ac92d1078c8b03b92ca7efb8 WatchSource:0}: Error finding container dd7baac25a0f4b4ff95ae3aa62ee8eeb8d22b675ac92d1078c8b03b92ca7efb8: Status 404 returned error can't find the container with id dd7baac25a0f4b4ff95ae3aa62ee8eeb8d22b675ac92d1078c8b03b92ca7efb8
Jul 07 14:25:32.157241 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.157201    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" event={"ID":"dda11bab-431b-4acd-b3c1-bc0b57459f9a","Type":"ContainerStarted","Data":"20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13"}
Jul 07 14:25:32.157405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.157270    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:32.157405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.157302    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:32.157405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.157370    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" event={"ID":"dda11bab-431b-4acd-b3c1-bc0b57459f9a","Type":"ContainerStarted","Data":"5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054"}
Jul 07 14:25:32.157554 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.157412    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" event={"ID":"dda11bab-431b-4acd-b3c1-bc0b57459f9a","Type":"ContainerStarted","Data":"dd7baac25a0f4b4ff95ae3aa62ee8eeb8d22b675ac92d1078c8b03b92ca7efb8"}
Jul 07 14:25:32.158625 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.158598    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:25:32.176599 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:32.176517    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podStartSLOduration=1.176506661 podStartE2EDuration="1.176506661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:25:32.176374209 +0000 UTC m=+986.465283306" watchObservedRunningTime="2026-07-07 14:25:32.176506661 +0000 UTC m=+986.465415759"
Jul 07 14:25:33.161655 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:33.161613    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:25:38.166073 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:38.166049    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:25:38.166473 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:38.166450    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:25:48.166905 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:48.166868    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:25:58.166452 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:25:58.166411    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:26:08.166469 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:26:08.166432    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:26:18.167185 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:26:18.167134    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:29:06.183186 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:29:06.183159    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:29:06.186134 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:29:06.186109    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:33:56.144606 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.144577    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"]
Jul 07 14:33:56.148200 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.148181    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.150714 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.150695    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-03875-predictor-serving-cert\""
Jul 07 14:33:56.150802 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.150703    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-03875-kube-rbac-proxy-sar-config\""
Jul 07 14:33:56.157485 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.157461    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"]
Jul 07 14:33:56.194986 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.194957    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"]
Jul 07 14:33:56.195283 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.195254    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container" containerID="cri-o://6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649" gracePeriod=30
Jul 07 14:33:56.195406 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.195346    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kube-rbac-proxy" containerID="cri-o://de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120" gracePeriod=30
Jul 07 14:33:56.233530 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.233507    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhkn\" (UniqueName: \"kubernetes.io/projected/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-api-access-fdhkn\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.233651 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.233552    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.233651 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.233586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.334121 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.334098    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhkn\" (UniqueName: \"kubernetes.io/projected/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-api-access-fdhkn\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.334261 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.334176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.334261 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.334223    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.334510 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:33:56.334487    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-03875-predictor-serving-cert: secret "error-404-isvc-03875-predictor-serving-cert" not found
Jul 07 14:33:56.334608 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:33:56.334555    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls podName:68ee556b-1eac-454f-a609-5dd6c6de0cc9 nodeName:}" failed. No retries permitted until 2026-07-07 14:33:56.834536679 +0000 UTC m=+1491.123445757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls") pod "error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" (UID: "68ee556b-1eac-454f-a609-5dd6c6de0cc9") : secret "error-404-isvc-03875-predictor-serving-cert" not found
Jul 07 14:33:56.334812 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.334795    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.342477 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.342455    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhkn\" (UniqueName: \"kubernetes.io/projected/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-api-access-fdhkn\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.459859 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.459804    2570 generic.go:356] "Generic (PLEG): container finished" podID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerID="de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120" exitCode=2
Jul 07 14:33:56.459943 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.459872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" event={"ID":"3b481e4c-1994-4977-a83d-aec0e20ab4ca","Type":"ContainerDied","Data":"de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120"}
Jul 07 14:33:56.838636 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.838559    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:56.840917 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:56.840896    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls\") pod \"error-404-isvc-03875-predictor-7f99c97bbb-2kzgp\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") " pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:57.061625 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.061595    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:57.189625 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.189604    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"]
Jul 07 14:33:57.191435 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:33:57.191407    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ee556b_1eac_454f_a609_5dd6c6de0cc9.slice/crio-aa5702e20ab12a3e1849cc7f111e3b136e403b34febd9168aacf0483cea0ed49 WatchSource:0}: Error finding container aa5702e20ab12a3e1849cc7f111e3b136e403b34febd9168aacf0483cea0ed49: Status 404 returned error can't find the container with id aa5702e20ab12a3e1849cc7f111e3b136e403b34febd9168aacf0483cea0ed49
Jul 07 14:33:57.466587 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.466518    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" event={"ID":"68ee556b-1eac-454f-a609-5dd6c6de0cc9","Type":"ContainerStarted","Data":"a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b"}
Jul 07 14:33:57.466700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.466587    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:57.466700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.466613    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" event={"ID":"68ee556b-1eac-454f-a609-5dd6c6de0cc9","Type":"ContainerStarted","Data":"5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9"}
Jul 07 14:33:57.466700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.466636    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:33:57.466700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.466655    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" event={"ID":"68ee556b-1eac-454f-a609-5dd6c6de0cc9","Type":"ContainerStarted","Data":"aa5702e20ab12a3e1849cc7f111e3b136e403b34febd9168aacf0483cea0ed49"}
Jul 07 14:33:57.468067 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.468040    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 07 14:33:57.489440 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:57.489392    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podStartSLOduration=1.4893773160000001 podStartE2EDuration="1.489377316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:57.486536622 +0000 UTC m=+1491.775445720" watchObservedRunningTime="2026-07-07 14:33:57.489377316 +0000 UTC m=+1491.778286413"
Jul 07 14:33:58.470936 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.470887    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 07 14:33:58.842460 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.842439    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:33:58.857353 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.857327    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq8sf\" (UniqueName: \"kubernetes.io/projected/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-api-access-xq8sf\") pod \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") "
Jul 07 14:33:58.857461 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.857379    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b481e4c-1994-4977-a83d-aec0e20ab4ca-proxy-tls\") pod \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") "
Jul 07 14:33:58.857461 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.857413    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-rbac-proxy-sar-config\") pod \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\" (UID: \"3b481e4c-1994-4977-a83d-aec0e20ab4ca\") "
Jul 07 14:33:58.857684 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.857663    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3b481e4c-1994-4977-a83d-aec0e20ab4ca" (UID: "3b481e4c-1994-4977-a83d-aec0e20ab4ca"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:33:58.859271 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.859251    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b481e4c-1994-4977-a83d-aec0e20ab4ca-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3b481e4c-1994-4977-a83d-aec0e20ab4ca" (UID: "3b481e4c-1994-4977-a83d-aec0e20ab4ca"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:33:58.859346 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.859331    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-api-access-xq8sf" (OuterVolumeSpecName: "kube-api-access-xq8sf") pod "3b481e4c-1994-4977-a83d-aec0e20ab4ca" (UID: "3b481e4c-1994-4977-a83d-aec0e20ab4ca"). InnerVolumeSpecName "kube-api-access-xq8sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:33:58.958826 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.958802    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq8sf\" (UniqueName: \"kubernetes.io/projected/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-api-access-xq8sf\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:33:58.958826 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.958822    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b481e4c-1994-4977-a83d-aec0e20ab4ca-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:33:58.958948 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:58.958833    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3b481e4c-1994-4977-a83d-aec0e20ab4ca-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:33:59.476842 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.476809    2570 generic.go:356] "Generic (PLEG): container finished" podID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerID="6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649" exitCode=0
Jul 07 14:33:59.477305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.476877    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"
Jul 07 14:33:59.477305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.476892    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" event={"ID":"3b481e4c-1994-4977-a83d-aec0e20ab4ca","Type":"ContainerDied","Data":"6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649"}
Jul 07 14:33:59.477305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.476949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv" event={"ID":"3b481e4c-1994-4977-a83d-aec0e20ab4ca","Type":"ContainerDied","Data":"4416b413ff97ec54a3108afaa329054205a5c935ffce97dd1bad4c24cd74e69b"}
Jul 07 14:33:59.477305 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.476983    2570 scope.go:117] "RemoveContainer" containerID="de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120"
Jul 07 14:33:59.486623 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.486607    2570 scope.go:117] "RemoveContainer" containerID="6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649"
Jul 07 14:33:59.494405 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.494391    2570 scope.go:117] "RemoveContainer" containerID="de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120"
Jul 07 14:33:59.494650 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:33:59.494630    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120\": container with ID starting with de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120 not found: ID does not exist" containerID="de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120"
Jul 07 14:33:59.494691 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.494658    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120"} err="failed to get container status \"de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120\": rpc error: code = NotFound desc = could not find container \"de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120\": container with ID starting with de3082fce03f0e0e52c5940e652284faad71a5703a843db1a3b7137c0ce86120 not found: ID does not exist"
Jul 07 14:33:59.494691 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.494675    2570 scope.go:117] "RemoveContainer" containerID="6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649"
Jul 07 14:33:59.494896 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:33:59.494880    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649\": container with ID starting with 6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649 not found: ID does not exist" containerID="6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649"
Jul 07 14:33:59.494937 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.494903    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649"} err="failed to get container status \"6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649\": rpc error: code = NotFound desc = could not find container \"6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649\": container with ID starting with 6eb95286e3c3acc2af429158769db8490310b5ddd111abd3f614819b95007649 not found: ID does not exist"
Jul 07 14:33:59.499700 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.499681    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"]
Jul 07 14:33:59.503374 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:33:59.503357    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-520df-predictor-696bbdd946-ns2nv"]
Jul 07 14:34:00.242093 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:00.242063    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" path="/var/lib/kubelet/pods/3b481e4c-1994-4977-a83d-aec0e20ab4ca/volumes"
Jul 07 14:34:03.475878 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:03.475853    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:34:03.476314 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:03.476268    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 07 14:34:06.217491 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:06.217464    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:34:06.220063 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:06.220040    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:34:13.476832 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:13.476792    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 07 14:34:23.476472 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:23.476421    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 07 14:34:33.476979 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:33.476938    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused"
Jul 07 14:34:43.477267 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:43.477235    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:34:45.857447 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.857413    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"]
Jul 07 14:34:45.857924 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.857905    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container"
Jul 07 14:34:45.857998 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.857928    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container"
Jul 07 14:34:45.857998 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.857947    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kube-rbac-proxy"
Jul 07 14:34:45.857998 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.857956    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kube-rbac-proxy"
Jul 07 14:34:45.858139 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.858061    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kserve-container"
Jul 07 14:34:45.858139 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.858076    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b481e4c-1994-4977-a83d-aec0e20ab4ca" containerName="kube-rbac-proxy"
Jul 07 14:34:45.861256 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.861230    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:45.863673 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.863652    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fbc26-predictor-serving-cert\""
Jul 07 14:34:45.864087 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.864070    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fbc26-kube-rbac-proxy-sar-config\""
Jul 07 14:34:45.874962 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.874939    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"]
Jul 07 14:34:45.911685 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.911658    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"]
Jul 07 14:34:45.911933 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.911901    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" containerID="cri-o://5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054" gracePeriod=30
Jul 07 14:34:45.912184 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.911953    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kube-rbac-proxy" containerID="cri-o://20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13" gracePeriod=30
Jul 07 14:34:45.999915 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.999892    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.000012 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:45.999933    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxp8q\" (UniqueName: \"kubernetes.io/projected/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-api-access-mxp8q\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.000012 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.000004    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.100843 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.100819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.100934 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.100855    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxp8q\" (UniqueName: \"kubernetes.io/projected/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-api-access-mxp8q\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.100934 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.100886    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.101110 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:34:46.101088    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-serving-cert: secret "error-404-isvc-fbc26-predictor-serving-cert" not found
Jul 07 14:34:46.101184 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:34:46.101177    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls podName:e3dba091-791a-4f6a-9d67-881370dd1aa2 nodeName:}" failed. No retries permitted until 2026-07-07 14:34:46.601135391 +0000 UTC m=+1540.890044468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls") pod "error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" (UID: "e3dba091-791a-4f6a-9d67-881370dd1aa2") : secret "error-404-isvc-fbc26-predictor-serving-cert" not found
Jul 07 14:34:46.101534 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.101517    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.109140 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.109085    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxp8q\" (UniqueName: \"kubernetes.io/projected/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-api-access-mxp8q\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.604523 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.604500    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.606813 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.606787    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls\") pod \"error-404-isvc-fbc26-predictor-78ddb55798-rjlzh\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") " pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.703314 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.703283    2570 generic.go:356] "Generic (PLEG): container finished" podID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerID="20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13" exitCode=2
Jul 07 14:34:46.703435 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.703353    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" event={"ID":"dda11bab-431b-4acd-b3c1-bc0b57459f9a","Type":"ContainerDied","Data":"20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13"}
Jul 07 14:34:46.771750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.771729    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:46.905171 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:46.905135    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"]
Jul 07 14:34:46.907233 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:34:46.907211    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3dba091_791a_4f6a_9d67_881370dd1aa2.slice/crio-aa5a406b32355898e836bd90b713f48efdbf0d25fca689845a99b351da7aef90 WatchSource:0}: Error finding container aa5a406b32355898e836bd90b713f48efdbf0d25fca689845a99b351da7aef90: Status 404 returned error can't find the container with id aa5a406b32355898e836bd90b713f48efdbf0d25fca689845a99b351da7aef90
Jul 07 14:34:47.710654 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.710615    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" event={"ID":"e3dba091-791a-4f6a-9d67-881370dd1aa2","Type":"ContainerStarted","Data":"3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556"}
Jul 07 14:34:47.710851 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.710710    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:47.710851 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.710739    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" event={"ID":"e3dba091-791a-4f6a-9d67-881370dd1aa2","Type":"ContainerStarted","Data":"9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c"}
Jul 07 14:34:47.710851 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.710761    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" event={"ID":"e3dba091-791a-4f6a-9d67-881370dd1aa2","Type":"ContainerStarted","Data":"aa5a406b32355898e836bd90b713f48efdbf0d25fca689845a99b351da7aef90"}
Jul 07 14:34:47.710851 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.710780    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:47.712223 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.712198    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 14:34:47.730672 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:47.730626    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podStartSLOduration=2.730614129 podStartE2EDuration="2.730614129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:34:47.730256454 +0000 UTC m=+1542.019165551" watchObservedRunningTime="2026-07-07 14:34:47.730614129 +0000 UTC m=+1542.019523282"
Jul 07 14:34:48.162470 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.162433    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.34:8643/healthz\": dial tcp 10.132.0.34:8643: connect: connection refused"
Jul 07 14:34:48.166731 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.166699    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Jul 07 14:34:48.456423 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.456398    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:34:48.619824 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.619792    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dda11bab-431b-4acd-b3c1-bc0b57459f9a-proxy-tls\") pod \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") "
Jul 07 14:34:48.619945 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.619841    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-rbac-proxy-sar-config\") pod \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") "
Jul 07 14:34:48.619945 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.619899    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq8pw\" (UniqueName: \"kubernetes.io/projected/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-api-access-qq8pw\") pod \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\" (UID: \"dda11bab-431b-4acd-b3c1-bc0b57459f9a\") "
Jul 07 14:34:48.620298 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.620270    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "dda11bab-431b-4acd-b3c1-bc0b57459f9a" (UID: "dda11bab-431b-4acd-b3c1-bc0b57459f9a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:34:48.621844 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.621821    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda11bab-431b-4acd-b3c1-bc0b57459f9a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dda11bab-431b-4acd-b3c1-bc0b57459f9a" (UID: "dda11bab-431b-4acd-b3c1-bc0b57459f9a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:34:48.621901 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.621865    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-api-access-qq8pw" (OuterVolumeSpecName: "kube-api-access-qq8pw") pod "dda11bab-431b-4acd-b3c1-bc0b57459f9a" (UID: "dda11bab-431b-4acd-b3c1-bc0b57459f9a"). InnerVolumeSpecName "kube-api-access-qq8pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:34:48.715804 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.715744    2570 generic.go:356] "Generic (PLEG): container finished" podID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerID="5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054" exitCode=0
Jul 07 14:34:48.715804 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.715794    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"
Jul 07 14:34:48.715911 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.715826    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" event={"ID":"dda11bab-431b-4acd-b3c1-bc0b57459f9a","Type":"ContainerDied","Data":"5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054"}
Jul 07 14:34:48.715911 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.715889    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr" event={"ID":"dda11bab-431b-4acd-b3c1-bc0b57459f9a","Type":"ContainerDied","Data":"dd7baac25a0f4b4ff95ae3aa62ee8eeb8d22b675ac92d1078c8b03b92ca7efb8"}
Jul 07 14:34:48.715971 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.715918    2570 scope.go:117] "RemoveContainer" containerID="20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13"
Jul 07 14:34:48.716488 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.716460    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 14:34:48.720768 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.720748    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dda11bab-431b-4acd-b3c1-bc0b57459f9a-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:34:48.720846 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.720772    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:34:48.720846 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.720786    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq8pw\" (UniqueName: \"kubernetes.io/projected/dda11bab-431b-4acd-b3c1-bc0b57459f9a-kube-api-access-qq8pw\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:34:48.724256 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.724233    2570 scope.go:117] "RemoveContainer" containerID="5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054"
Jul 07 14:34:48.731221 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.731206    2570 scope.go:117] "RemoveContainer" containerID="20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13"
Jul 07 14:34:48.731465 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:34:48.731449    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13\": container with ID starting with 20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13 not found: ID does not exist" containerID="20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13"
Jul 07 14:34:48.731524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.731472    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13"} err="failed to get container status \"20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13\": rpc error: code = NotFound desc = could not find container \"20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13\": container with ID starting with 20d4dea155e3946a0b56595359373e13335b1a490283be1d2aab43f040555f13 not found: ID does not exist"
Jul 07 14:34:48.731524 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.731488    2570 scope.go:117] "RemoveContainer" containerID="5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054"
Jul 07 14:34:48.731706 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:34:48.731690    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054\": container with ID starting with 5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054 not found: ID does not exist" containerID="5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054"
Jul 07 14:34:48.731750 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.731711    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054"} err="failed to get container status \"5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054\": rpc error: code = NotFound desc = could not find container \"5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054\": container with ID starting with 5bf4b8bd98bce395f4868ceb2d47dfa112d36121c37a697547e8d13c49772054 not found: ID does not exist"
Jul 07 14:34:48.738443 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.738424    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"]
Jul 07 14:34:48.741929 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:48.741912    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ab362-predictor-56447f4fd8-2k4qr"]
Jul 07 14:34:50.241421 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:50.241385    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" path="/var/lib/kubelet/pods/dda11bab-431b-4acd-b3c1-bc0b57459f9a/volumes"
Jul 07 14:34:53.720121 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:53.720094    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:34:53.720618 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:34:53.720558    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 14:35:03.721606 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:03.721568    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 14:35:06.412877 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.412822    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"]
Jul 07 14:35:06.413311 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.413226    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kube-rbac-proxy"
Jul 07 14:35:06.413311 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.413241    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kube-rbac-proxy"
Jul 07 14:35:06.413311 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.413262    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container"
Jul 07 14:35:06.413311 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.413267    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container"
Jul 07 14:35:06.413515 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.413336    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kserve-container"
Jul 07 14:35:06.413515 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.413347    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda11bab-431b-4acd-b3c1-bc0b57459f9a" containerName="kube-rbac-proxy"
Jul 07 14:35:06.418088 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.418068    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.420268 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.420247    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fe15b-predictor-serving-cert\""
Jul 07 14:35:06.420375 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.420268    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-fe15b-kube-rbac-proxy-sar-config\""
Jul 07 14:35:06.428624 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.428601    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"]
Jul 07 14:35:06.467112 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.467086    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"]
Jul 07 14:35:06.467399 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.467374    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container" containerID="cri-o://5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9" gracePeriod=30
Jul 07 14:35:06.467608 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.467390    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kube-rbac-proxy" containerID="cri-o://a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b" gracePeriod=30
Jul 07 14:35:06.557591 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.557528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.557689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.557624    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.557689 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.557656    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skzqz\" (UniqueName: \"kubernetes.io/projected/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-api-access-skzqz\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.658294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.658268    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.658423 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.658308    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skzqz\" (UniqueName: \"kubernetes.io/projected/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-api-access-skzqz\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.658423 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.658375    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.658530 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:35:06.658426    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-serving-cert: secret "error-404-isvc-fe15b-predictor-serving-cert" not found
Jul 07 14:35:06.658530 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:35:06.658490    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls podName:824450cf-0ca7-41f8-9099-041a11e42b0a nodeName:}" failed. No retries permitted until 2026-07-07 14:35:07.158474373 +0000 UTC m=+1561.447383449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls") pod "error-404-isvc-fe15b-predictor-957bff694-hw6p5" (UID: "824450cf-0ca7-41f8-9099-041a11e42b0a") : secret "error-404-isvc-fe15b-predictor-serving-cert" not found
Jul 07 14:35:06.658972 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.658955    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.668818 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.668788    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skzqz\" (UniqueName: \"kubernetes.io/projected/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-api-access-skzqz\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:06.806302 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.806275    2570 generic.go:356] "Generic (PLEG): container finished" podID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerID="a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b" exitCode=2
Jul 07 14:35:06.806450 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:06.806332    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" event={"ID":"68ee556b-1eac-454f-a609-5dd6c6de0cc9","Type":"ContainerDied","Data":"a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b"}
Jul 07 14:35:07.162430 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.162399    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:07.164746 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.164715    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls\") pod \"error-404-isvc-fe15b-predictor-957bff694-hw6p5\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") " pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:07.329592 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.329564    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:07.460904 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.460837    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"]
Jul 07 14:35:07.463778 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:35:07.463748    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824450cf_0ca7_41f8_9099_041a11e42b0a.slice/crio-4d7118766501f989573648460be4a7cacae85a9bc5fc6b715c567e980e900371 WatchSource:0}: Error finding container 4d7118766501f989573648460be4a7cacae85a9bc5fc6b715c567e980e900371: Status 404 returned error can't find the container with id 4d7118766501f989573648460be4a7cacae85a9bc5fc6b715c567e980e900371
Jul 07 14:35:07.812928 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.812854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" event={"ID":"824450cf-0ca7-41f8-9099-041a11e42b0a","Type":"ContainerStarted","Data":"e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb"}
Jul 07 14:35:07.812928 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.812908    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" event={"ID":"824450cf-0ca7-41f8-9099-041a11e42b0a","Type":"ContainerStarted","Data":"ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa"}
Jul 07 14:35:07.813115 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.812943    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:07.813115 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.813094    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:07.813193 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.813140    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" event={"ID":"824450cf-0ca7-41f8-9099-041a11e42b0a","Type":"ContainerStarted","Data":"4d7118766501f989573648460be4a7cacae85a9bc5fc6b715c567e980e900371"}
Jul 07 14:35:07.814264 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.814243    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:35:07.833017 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:07.832979    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podStartSLOduration=1.832966334 podStartE2EDuration="1.832966334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:07.831954733 +0000 UTC m=+1562.120864024" watchObservedRunningTime="2026-07-07 14:35:07.832966334 +0000 UTC m=+1562.121875432"
Jul 07 14:35:08.471390 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:08.471352    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused"
Jul 07 14:35:08.817701 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:08.817611    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:35:09.221216 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.221193    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:35:09.380020 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.379996    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls\") pod \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") "
Jul 07 14:35:09.380193 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.380071    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-rbac-proxy-sar-config\") pod \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") "
Jul 07 14:35:09.380193 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.380125    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdhkn\" (UniqueName: \"kubernetes.io/projected/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-api-access-fdhkn\") pod \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\" (UID: \"68ee556b-1eac-454f-a609-5dd6c6de0cc9\") "
Jul 07 14:35:09.380503 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.380475    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "68ee556b-1eac-454f-a609-5dd6c6de0cc9" (UID: "68ee556b-1eac-454f-a609-5dd6c6de0cc9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:35:09.382528 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.382497    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "68ee556b-1eac-454f-a609-5dd6c6de0cc9" (UID: "68ee556b-1eac-454f-a609-5dd6c6de0cc9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:35:09.382849 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.382814    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-api-access-fdhkn" (OuterVolumeSpecName: "kube-api-access-fdhkn") pod "68ee556b-1eac-454f-a609-5dd6c6de0cc9" (UID: "68ee556b-1eac-454f-a609-5dd6c6de0cc9"). InnerVolumeSpecName "kube-api-access-fdhkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:35:09.481484 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.481463    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdhkn\" (UniqueName: \"kubernetes.io/projected/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-api-access-fdhkn\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:35:09.481484 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.481485    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/68ee556b-1eac-454f-a609-5dd6c6de0cc9-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:35:09.481777 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.481494    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/68ee556b-1eac-454f-a609-5dd6c6de0cc9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:35:09.823173 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.823091    2570 generic.go:356] "Generic (PLEG): container finished" podID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerID="5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9" exitCode=0
Jul 07 14:35:09.823173 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.823137    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"
Jul 07 14:35:09.823322 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.823175    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" event={"ID":"68ee556b-1eac-454f-a609-5dd6c6de0cc9","Type":"ContainerDied","Data":"5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9"}
Jul 07 14:35:09.823322 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.823239    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp" event={"ID":"68ee556b-1eac-454f-a609-5dd6c6de0cc9","Type":"ContainerDied","Data":"aa5702e20ab12a3e1849cc7f111e3b136e403b34febd9168aacf0483cea0ed49"}
Jul 07 14:35:09.823322 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.823268    2570 scope.go:117] "RemoveContainer" containerID="a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b"
Jul 07 14:35:09.832308 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.832289    2570 scope.go:117] "RemoveContainer" containerID="5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9"
Jul 07 14:35:09.841450 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.841436    2570 scope.go:117] "RemoveContainer" containerID="a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b"
Jul 07 14:35:09.841681 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:35:09.841665    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b\": container with ID starting with a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b not found: ID does not exist" containerID="a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b"
Jul 07 14:35:09.841718 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.841689    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b"} err="failed to get container status \"a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b\": rpc error: code = NotFound desc = could not find container \"a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b\": container with ID starting with a95b24e6bd9ba9879401391527941e0c5f9ea64eb0a8a07eded565559cd61e0b not found: ID does not exist"
Jul 07 14:35:09.841718 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.841703    2570 scope.go:117] "RemoveContainer" containerID="5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9"
Jul 07 14:35:09.841903 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:35:09.841890    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9\": container with ID starting with 5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9 not found: ID does not exist" containerID="5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9"
Jul 07 14:35:09.841940 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.841908    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9"} err="failed to get container status \"5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9\": rpc error: code = NotFound desc = could not find container \"5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9\": container with ID starting with 5c79b3e85504df334f7e070578aecd2786bfa0bcf3c89729a466cae54cd21cb9 not found: ID does not exist"
Jul 07 14:35:09.844796 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.844776    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"]
Jul 07 14:35:09.848622 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:09.848603    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-03875-predictor-7f99c97bbb-2kzgp"]
Jul 07 14:35:10.241368 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:10.241344    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" path="/var/lib/kubelet/pods/68ee556b-1eac-454f-a609-5dd6c6de0cc9/volumes"
Jul 07 14:35:13.721063 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:13.721028    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 14:35:13.821645 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:13.821623    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:13.822007 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:13.821984    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:35:23.721407 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:23.721358    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused"
Jul 07 14:35:23.822041 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:23.822004    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:35:33.721632 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:33.721598    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:35:33.822869 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:33.822835    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:35:43.822799 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:43.822759    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:35:53.823112 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:53.823076    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:35:55.957295 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:55.957265    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"]
Jul 07 14:35:55.957659 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:55.957494    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container" containerID="cri-o://9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c" gracePeriod=30
Jul 07 14:35:55.957659 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:55.957504    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kube-rbac-proxy" containerID="cri-o://3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556" gracePeriod=30
Jul 07 14:35:56.125730 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.125701    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"]
Jul 07 14:35:56.126144 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.126128    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kube-rbac-proxy"
Jul 07 14:35:56.126224 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.126161    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kube-rbac-proxy"
Jul 07 14:35:56.126224 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.126176    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container"
Jul 07 14:35:56.126224 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.126184    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container"
Jul 07 14:35:56.126372 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.126275    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kserve-container"
Jul 07 14:35:56.126372 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.126287    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="68ee556b-1eac-454f-a609-5dd6c6de0cc9" containerName="kube-rbac-proxy"
Jul 07 14:35:56.129376 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.129350    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.132062 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.132040    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f31ce-kube-rbac-proxy-sar-config\""
Jul 07 14:35:56.132192 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.132047    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f31ce-predictor-serving-cert\""
Jul 07 14:35:56.140623 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.140601    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"]
Jul 07 14:35:56.214931 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.214873    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.214931 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.214903    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkgz7\" (UniqueName: \"kubernetes.io/projected/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-api-access-jkgz7\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.215052 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.214932    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/480cce6f-bfb8-4409-b5bd-38c19c774e6c-proxy-tls\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.316356 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.316333    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.316464 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.316365    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkgz7\" (UniqueName: \"kubernetes.io/projected/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-api-access-jkgz7\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.316464 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.316400    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/480cce6f-bfb8-4409-b5bd-38c19c774e6c-proxy-tls\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.317035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.317010    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.318789 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.318772    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/480cce6f-bfb8-4409-b5bd-38c19c774e6c-proxy-tls\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.326637 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.326618    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkgz7\" (UniqueName: \"kubernetes.io/projected/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-api-access-jkgz7\") pod \"error-404-isvc-f31ce-predictor-85665c44b4-nq6c7\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") " pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.443130 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.443110    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:56.576340 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:56.576315    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"]
Jul 07 14:35:56.578083 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:35:56.578055    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod480cce6f_bfb8_4409_b5bd_38c19c774e6c.slice/crio-2bb905b8c7576afa1a99ba9a0cf34655cd1217da5fb69575964d2c32e715a58c WatchSource:0}: Error finding container 2bb905b8c7576afa1a99ba9a0cf34655cd1217da5fb69575964d2c32e715a58c: Status 404 returned error can't find the container with id 2bb905b8c7576afa1a99ba9a0cf34655cd1217da5fb69575964d2c32e715a58c
Jul 07 14:35:57.049657 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.049626    2570 generic.go:356] "Generic (PLEG): container finished" podID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerID="3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556" exitCode=2
Jul 07 14:35:57.050042 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.049697    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" event={"ID":"e3dba091-791a-4f6a-9d67-881370dd1aa2","Type":"ContainerDied","Data":"3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556"}
Jul 07 14:35:57.051262 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.051242    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" event={"ID":"480cce6f-bfb8-4409-b5bd-38c19c774e6c","Type":"ContainerStarted","Data":"f87222f82df705c102226a5afc5c747047cb9c3ff4a6985e305006b2f97eaf90"}
Jul 07 14:35:57.051356 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.051286    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" event={"ID":"480cce6f-bfb8-4409-b5bd-38c19c774e6c","Type":"ContainerStarted","Data":"f56aa890c82f38e7a3b76d7583ff099cd20d7e0189f78ee04866746c7fe0d356"}
Jul 07 14:35:57.051356 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.051310    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" event={"ID":"480cce6f-bfb8-4409-b5bd-38c19c774e6c","Type":"ContainerStarted","Data":"2bb905b8c7576afa1a99ba9a0cf34655cd1217da5fb69575964d2c32e715a58c"}
Jul 07 14:35:57.051356 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.051335    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:57.051446 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.051372    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:35:57.052699 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.052678    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:35:57.070960 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:57.070924    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podStartSLOduration=1.070910633 podStartE2EDuration="1.070910633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:57.070143773 +0000 UTC m=+1611.359052872" watchObservedRunningTime="2026-07-07 14:35:57.070910633 +0000 UTC m=+1611.359819730"
Jul 07 14:35:58.055874 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.055787    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:35:58.716537 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.716493    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused"
Jul 07 14:35:58.797041 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.797020    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:35:58.939963 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.939939    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls\") pod \"e3dba091-791a-4f6a-9d67-881370dd1aa2\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") "
Jul 07 14:35:58.940103 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.939970    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxp8q\" (UniqueName: \"kubernetes.io/projected/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-api-access-mxp8q\") pod \"e3dba091-791a-4f6a-9d67-881370dd1aa2\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") "
Jul 07 14:35:58.940103 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.940022    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-rbac-proxy-sar-config\") pod \"e3dba091-791a-4f6a-9d67-881370dd1aa2\" (UID: \"e3dba091-791a-4f6a-9d67-881370dd1aa2\") "
Jul 07 14:35:58.940400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.940372    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e3dba091-791a-4f6a-9d67-881370dd1aa2" (UID: "e3dba091-791a-4f6a-9d67-881370dd1aa2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:35:58.941987 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.941964    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e3dba091-791a-4f6a-9d67-881370dd1aa2" (UID: "e3dba091-791a-4f6a-9d67-881370dd1aa2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:35:58.942049 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:58.942017    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-api-access-mxp8q" (OuterVolumeSpecName: "kube-api-access-mxp8q") pod "e3dba091-791a-4f6a-9d67-881370dd1aa2" (UID: "e3dba091-791a-4f6a-9d67-881370dd1aa2"). InnerVolumeSpecName "kube-api-access-mxp8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:35:59.041331 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.041309    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e3dba091-791a-4f6a-9d67-881370dd1aa2-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:35:59.041331 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.041330    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxp8q\" (UniqueName: \"kubernetes.io/projected/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-api-access-mxp8q\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:35:59.041454 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.041342    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e3dba091-791a-4f6a-9d67-881370dd1aa2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:35:59.062212 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.062181    2570 generic.go:356] "Generic (PLEG): container finished" podID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerID="9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c" exitCode=0
Jul 07 14:35:59.062550 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.062233    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"
Jul 07 14:35:59.062550 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.062261    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" event={"ID":"e3dba091-791a-4f6a-9d67-881370dd1aa2","Type":"ContainerDied","Data":"9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c"}
Jul 07 14:35:59.062550 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.062340    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh" event={"ID":"e3dba091-791a-4f6a-9d67-881370dd1aa2","Type":"ContainerDied","Data":"aa5a406b32355898e836bd90b713f48efdbf0d25fca689845a99b351da7aef90"}
Jul 07 14:35:59.062550 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.062378    2570 scope.go:117] "RemoveContainer" containerID="3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556"
Jul 07 14:35:59.071017 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.071001    2570 scope.go:117] "RemoveContainer" containerID="9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c"
Jul 07 14:35:59.080042 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.079259    2570 scope.go:117] "RemoveContainer" containerID="3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556"
Jul 07 14:35:59.080540 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:35:59.080520    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556\": container with ID starting with 3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556 not found: ID does not exist" containerID="3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556"
Jul 07 14:35:59.080619 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.080544    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556"} err="failed to get container status \"3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556\": rpc error: code = NotFound desc = could not find container \"3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556\": container with ID starting with 3474578dbc492e101b076e7084af5b02dcb89dab51fae296b8886b4583e82556 not found: ID does not exist"
Jul 07 14:35:59.080619 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.080559    2570 scope.go:117] "RemoveContainer" containerID="9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c"
Jul 07 14:35:59.080825 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:35:59.080807    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c\": container with ID starting with 9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c not found: ID does not exist" containerID="9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c"
Jul 07 14:35:59.080883 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.080833    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c"} err="failed to get container status \"9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c\": rpc error: code = NotFound desc = could not find container \"9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c\": container with ID starting with 9b694c85cb7f9d553b5c71ebc00f5af19d58cca47b38594b44dcbc63ca644f7c not found: ID does not exist"
Jul 07 14:35:59.087788 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.087768    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"]
Jul 07 14:35:59.091826 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:35:59.091807    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fbc26-predictor-78ddb55798-rjlzh"]
Jul 07 14:36:00.241722 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:00.241689    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" path="/var/lib/kubelet/pods/e3dba091-791a-4f6a-9d67-881370dd1aa2/volumes"
Jul 07 14:36:03.059928 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:03.059900    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:36:03.060431 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:03.060405    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:36:13.061357 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:13.061307    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:36:23.060865 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:23.060821    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:36:33.060737 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:33.060691    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:36:43.061337 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:36:43.061307    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:39:06.248269 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:39:06.248241    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:39:06.251922 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:39:06.251900    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:44:06.285355 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:06.285320    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:44:06.289113 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:06.289085    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:44:21.362158 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362108    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"]
Jul 07 14:44:21.362662 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362647    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container"
Jul 07 14:44:21.362707 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362666    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container"
Jul 07 14:44:21.362707 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362697    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kube-rbac-proxy"
Jul 07 14:44:21.362707 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362704    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kube-rbac-proxy"
Jul 07 14:44:21.362797 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362786    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kserve-container"
Jul 07 14:44:21.362833 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.362801    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3dba091-791a-4f6a-9d67-881370dd1aa2" containerName="kube-rbac-proxy"
Jul 07 14:44:21.366168 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.366135    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.368304 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.368282    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6e67f-predictor-serving-cert\""
Jul 07 14:44:21.368386 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.368287    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-6e67f-kube-rbac-proxy-sar-config\""
Jul 07 14:44:21.376362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.376343    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"]
Jul 07 14:44:21.403826 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.403763    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"]
Jul 07 14:44:21.404035 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.403990    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" containerID="cri-o://ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa" gracePeriod=30
Jul 07 14:44:21.404475 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.404446    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kube-rbac-proxy" containerID="cri-o://e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb" gracePeriod=30
Jul 07 14:44:21.443904 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.443880    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.444010 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.443922    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6qmd\" (UniqueName: \"kubernetes.io/projected/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-api-access-z6qmd\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.444010 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.443971    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42b9e97d-579f-4172-be60-2e8a3b22930a-proxy-tls\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.544909 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.544884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42b9e97d-579f-4172-be60-2e8a3b22930a-proxy-tls\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.545006 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.544938    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.545006 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.544968    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6qmd\" (UniqueName: \"kubernetes.io/projected/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-api-access-z6qmd\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.545667 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.545646    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.547363 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.547341    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42b9e97d-579f-4172-be60-2e8a3b22930a-proxy-tls\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.552103 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.552083    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6qmd\" (UniqueName: \"kubernetes.io/projected/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-api-access-z6qmd\") pod \"error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") " pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.677927 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.677869    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:21.807298 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:21.807275    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"]
Jul 07 14:44:21.809276 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:44:21.809250    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b9e97d_579f_4172_be60_2e8a3b22930a.slice/crio-85de76446760b98100ad8bede87db48780348dfb68faf69984753e6b8d5f219c WatchSource:0}: Error finding container 85de76446760b98100ad8bede87db48780348dfb68faf69984753e6b8d5f219c: Status 404 returned error can't find the container with id 85de76446760b98100ad8bede87db48780348dfb68faf69984753e6b8d5f219c
Jul 07 14:44:22.411248 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.411216    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" event={"ID":"42b9e97d-579f-4172-be60-2e8a3b22930a","Type":"ContainerStarted","Data":"48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832"}
Jul 07 14:44:22.411684 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.411273    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" event={"ID":"42b9e97d-579f-4172-be60-2e8a3b22930a","Type":"ContainerStarted","Data":"91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625"}
Jul 07 14:44:22.411684 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.411303    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" event={"ID":"42b9e97d-579f-4172-be60-2e8a3b22930a","Type":"ContainerStarted","Data":"85de76446760b98100ad8bede87db48780348dfb68faf69984753e6b8d5f219c"}
Jul 07 14:44:22.411684 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.411337    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:22.411684 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.411369    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:22.413001 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.412962    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 14:44:22.413246 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.413222    2570 generic.go:356] "Generic (PLEG): container finished" podID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerID="e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb" exitCode=2
Jul 07 14:44:22.413309 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.413285    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" event={"ID":"824450cf-0ca7-41f8-9099-041a11e42b0a","Type":"ContainerDied","Data":"e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb"}
Jul 07 14:44:22.429063 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:22.429024    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podStartSLOduration=1.429013233 podStartE2EDuration="1.429013233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:44:22.428082833 +0000 UTC m=+2116.716991934" watchObservedRunningTime="2026-07-07 14:44:22.429013233 +0000 UTC m=+2116.717922331"
Jul 07 14:44:23.417809 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:23.417765    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 14:44:23.818857 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:23.818764    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused"
Jul 07 14:44:23.822030 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:23.822002    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused"
Jul 07 14:44:24.158884 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.158857    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:44:24.161942 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.161919    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skzqz\" (UniqueName: \"kubernetes.io/projected/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-api-access-skzqz\") pod \"824450cf-0ca7-41f8-9099-041a11e42b0a\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") "
Jul 07 14:44:24.162007 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.161986    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-rbac-proxy-sar-config\") pod \"824450cf-0ca7-41f8-9099-041a11e42b0a\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") "
Jul 07 14:44:24.162076 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.162044    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls\") pod \"824450cf-0ca7-41f8-9099-041a11e42b0a\" (UID: \"824450cf-0ca7-41f8-9099-041a11e42b0a\") "
Jul 07 14:44:24.162416 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.162391    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "824450cf-0ca7-41f8-9099-041a11e42b0a" (UID: "824450cf-0ca7-41f8-9099-041a11e42b0a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:44:24.163870 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.163844    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-api-access-skzqz" (OuterVolumeSpecName: "kube-api-access-skzqz") pod "824450cf-0ca7-41f8-9099-041a11e42b0a" (UID: "824450cf-0ca7-41f8-9099-041a11e42b0a"). InnerVolumeSpecName "kube-api-access-skzqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:44:24.163976 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.163937    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "824450cf-0ca7-41f8-9099-041a11e42b0a" (UID: "824450cf-0ca7-41f8-9099-041a11e42b0a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:44:24.263159 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.263129    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:44:24.263243 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.263169    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/824450cf-0ca7-41f8-9099-041a11e42b0a-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:44:24.263243 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.263184    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skzqz\" (UniqueName: \"kubernetes.io/projected/824450cf-0ca7-41f8-9099-041a11e42b0a-kube-api-access-skzqz\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:44:24.422975 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.422948    2570 generic.go:356] "Generic (PLEG): container finished" podID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerID="ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa" exitCode=0
Jul 07 14:44:24.423289 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.422995    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"
Jul 07 14:44:24.423289 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.423027    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" event={"ID":"824450cf-0ca7-41f8-9099-041a11e42b0a","Type":"ContainerDied","Data":"ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa"}
Jul 07 14:44:24.423289 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.423078    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5" event={"ID":"824450cf-0ca7-41f8-9099-041a11e42b0a","Type":"ContainerDied","Data":"4d7118766501f989573648460be4a7cacae85a9bc5fc6b715c567e980e900371"}
Jul 07 14:44:24.423289 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.423106    2570 scope.go:117] "RemoveContainer" containerID="e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb"
Jul 07 14:44:24.431174 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.431140    2570 scope.go:117] "RemoveContainer" containerID="ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa"
Jul 07 14:44:24.438562 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.438547    2570 scope.go:117] "RemoveContainer" containerID="e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb"
Jul 07 14:44:24.438814 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:44:24.438796    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb\": container with ID starting with e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb not found: ID does not exist" containerID="e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb"
Jul 07 14:44:24.438876 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.438820    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb"} err="failed to get container status \"e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb\": rpc error: code = NotFound desc = could not find container \"e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb\": container with ID starting with e9b3240b6e71de2116508aea1ff0921d75ff2bd299f3c3bb333e43d1afc810fb not found: ID does not exist"
Jul 07 14:44:24.438876 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.438836    2570 scope.go:117] "RemoveContainer" containerID="ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa"
Jul 07 14:44:24.439087 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:44:24.439043    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa\": container with ID starting with ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa not found: ID does not exist" containerID="ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa"
Jul 07 14:44:24.439163 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.439097    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa"} err="failed to get container status \"ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa\": rpc error: code = NotFound desc = could not find container \"ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa\": container with ID starting with ab700ef254cd56ddaa44e1a3aca9cf8c22b2e85f0555ac37d25e48c10345c1fa not found: ID does not exist"
Jul 07 14:44:24.440271 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.440253    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"]
Jul 07 14:44:24.441843 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:24.441826    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-fe15b-predictor-957bff694-hw6p5"]
Jul 07 14:44:26.241925 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:26.241888    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" path="/var/lib/kubelet/pods/824450cf-0ca7-41f8-9099-041a11e42b0a/volumes"
Jul 07 14:44:28.421990 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:28.421959    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:44:28.422362 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:28.422321    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 14:44:38.423352 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:38.423314    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 14:44:48.422974 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:48.422937    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 14:44:58.422675 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:44:58.422643    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused"
Jul 07 14:45:08.423334 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:08.423305    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:45:11.062958 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.062926    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"]
Jul 07 14:45:11.063400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.063283    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kube-rbac-proxy"
Jul 07 14:45:11.063400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.063295    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kube-rbac-proxy"
Jul 07 14:45:11.063400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.063313    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container"
Jul 07 14:45:11.063400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.063319    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container"
Jul 07 14:45:11.063400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.063373    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kserve-container"
Jul 07 14:45:11.063400 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.063382    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="824450cf-0ca7-41f8-9099-041a11e42b0a" containerName="kube-rbac-proxy"
Jul 07 14:45:11.066411 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.066395    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.068488 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.068467    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-36f00-predictor-serving-cert\""
Jul 07 14:45:11.068488 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.068480    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-36f00-kube-rbac-proxy-sar-config\""
Jul 07 14:45:11.077616 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.077592    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"]
Jul 07 14:45:11.108196 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.108166    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"]
Jul 07 14:45:11.108464 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.108419    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" containerID="cri-o://f56aa890c82f38e7a3b76d7583ff099cd20d7e0189f78ee04866746c7fe0d356" gracePeriod=30
Jul 07 14:45:11.108563 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.108450    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kube-rbac-proxy" containerID="cri-o://f87222f82df705c102226a5afc5c747047cb9c3ff4a6985e305006b2f97eaf90" gracePeriod=30
Jul 07 14:45:11.197865 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.197844    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.197968 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.197876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6vw6\" (UniqueName: \"kubernetes.io/projected/fface55e-8428-4b30-aef5-14f023cb75ae-kube-api-access-t6vw6\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.197968 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.197921    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fface55e-8428-4b30-aef5-14f023cb75ae-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.298770 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.298748    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.298858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.298782    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6vw6\" (UniqueName: \"kubernetes.io/projected/fface55e-8428-4b30-aef5-14f023cb75ae-kube-api-access-t6vw6\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.298858 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.298851    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fface55e-8428-4b30-aef5-14f023cb75ae-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.298964 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:45:11.298896    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-36f00-predictor-serving-cert: secret "error-404-isvc-36f00-predictor-serving-cert" not found
Jul 07 14:45:11.298964 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:45:11.298954    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls podName:fface55e-8428-4b30-aef5-14f023cb75ae nodeName:}" failed. No retries permitted until 2026-07-07 14:45:11.798934149 +0000 UTC m=+2166.087843231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls") pod "error-404-isvc-36f00-predictor-cb475dffd-22vxb" (UID: "fface55e-8428-4b30-aef5-14f023cb75ae") : secret "error-404-isvc-36f00-predictor-serving-cert" not found
Jul 07 14:45:11.299473 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.299456    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fface55e-8428-4b30-aef5-14f023cb75ae-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.307427 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.307401    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6vw6\" (UniqueName: \"kubernetes.io/projected/fface55e-8428-4b30-aef5-14f023cb75ae-kube-api-access-t6vw6\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.644026 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.643990    2570 generic.go:356] "Generic (PLEG): container finished" podID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerID="f87222f82df705c102226a5afc5c747047cb9c3ff4a6985e305006b2f97eaf90" exitCode=2
Jul 07 14:45:11.644188 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.644035    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" event={"ID":"480cce6f-bfb8-4409-b5bd-38c19c774e6c","Type":"ContainerDied","Data":"f87222f82df705c102226a5afc5c747047cb9c3ff4a6985e305006b2f97eaf90"}
Jul 07 14:45:11.803006 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.802982    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.805440 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.805415    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls\") pod \"error-404-isvc-36f00-predictor-cb475dffd-22vxb\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") " pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:11.977913 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:11.977834    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:12.109739 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.109714    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"]
Jul 07 14:45:12.112041 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:45:12.112014    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfface55e_8428_4b30_aef5_14f023cb75ae.slice/crio-b572a40f6f70ffac6abca0d981ce7ea3860b11ecd9a2405dd0240fe4c97ac829 WatchSource:0}: Error finding container b572a40f6f70ffac6abca0d981ce7ea3860b11ecd9a2405dd0240fe4c97ac829: Status 404 returned error can't find the container with id b572a40f6f70ffac6abca0d981ce7ea3860b11ecd9a2405dd0240fe4c97ac829
Jul 07 14:45:12.650485 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.650450    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" event={"ID":"fface55e-8428-4b30-aef5-14f023cb75ae","Type":"ContainerStarted","Data":"f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e"}
Jul 07 14:45:12.650660 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.650510    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" event={"ID":"fface55e-8428-4b30-aef5-14f023cb75ae","Type":"ContainerStarted","Data":"a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79"}
Jul 07 14:45:12.650660 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.650542    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:12.650660 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.650575    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" event={"ID":"fface55e-8428-4b30-aef5-14f023cb75ae","Type":"ContainerStarted","Data":"b572a40f6f70ffac6abca0d981ce7ea3860b11ecd9a2405dd0240fe4c97ac829"}
Jul 07 14:45:12.650660 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.650605    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:12.651983 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.651956    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 14:45:12.668911 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:12.668871    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podStartSLOduration=1.668857637 podStartE2EDuration="1.668857637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:45:12.667367771 +0000 UTC m=+2166.956276904" watchObservedRunningTime="2026-07-07 14:45:12.668857637 +0000 UTC m=+2166.957766796"
Jul 07 14:45:13.055971 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.055886    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused"
Jul 07 14:45:13.060434 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.060408    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused"
Jul 07 14:45:13.658259 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.658227    2570 generic.go:356] "Generic (PLEG): container finished" podID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerID="f56aa890c82f38e7a3b76d7583ff099cd20d7e0189f78ee04866746c7fe0d356" exitCode=0
Jul 07 14:45:13.658553 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.658298    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" event={"ID":"480cce6f-bfb8-4409-b5bd-38c19c774e6c","Type":"ContainerDied","Data":"f56aa890c82f38e7a3b76d7583ff099cd20d7e0189f78ee04866746c7fe0d356"}
Jul 07 14:45:13.658601 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.658582    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 14:45:13.751256 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.751236    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:45:13.818387 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.818361    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkgz7\" (UniqueName: \"kubernetes.io/projected/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-api-access-jkgz7\") pod \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") "
Jul 07 14:45:13.818496 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.818421    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-rbac-proxy-sar-config\") pod \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") "
Jul 07 14:45:13.818567 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.818497    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/480cce6f-bfb8-4409-b5bd-38c19c774e6c-proxy-tls\") pod \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\" (UID: \"480cce6f-bfb8-4409-b5bd-38c19c774e6c\") "
Jul 07 14:45:13.818766 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.818744    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "480cce6f-bfb8-4409-b5bd-38c19c774e6c" (UID: "480cce6f-bfb8-4409-b5bd-38c19c774e6c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:45:13.820372 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.820352    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/480cce6f-bfb8-4409-b5bd-38c19c774e6c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "480cce6f-bfb8-4409-b5bd-38c19c774e6c" (UID: "480cce6f-bfb8-4409-b5bd-38c19c774e6c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:45:13.820471 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.820454    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-api-access-jkgz7" (OuterVolumeSpecName: "kube-api-access-jkgz7") pod "480cce6f-bfb8-4409-b5bd-38c19c774e6c" (UID: "480cce6f-bfb8-4409-b5bd-38c19c774e6c"). InnerVolumeSpecName "kube-api-access-jkgz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:45:13.919376 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.919356    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:45:13.919376 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.919376    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/480cce6f-bfb8-4409-b5bd-38c19c774e6c-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:45:13.919511 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:13.919385    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkgz7\" (UniqueName: \"kubernetes.io/projected/480cce6f-bfb8-4409-b5bd-38c19c774e6c-kube-api-access-jkgz7\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:45:14.664475 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:14.664444    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7" event={"ID":"480cce6f-bfb8-4409-b5bd-38c19c774e6c","Type":"ContainerDied","Data":"2bb905b8c7576afa1a99ba9a0cf34655cd1217da5fb69575964d2c32e715a58c"}
Jul 07 14:45:14.664475 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:14.664465    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"
Jul 07 14:45:14.664931 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:14.664510    2570 scope.go:117] "RemoveContainer" containerID="f87222f82df705c102226a5afc5c747047cb9c3ff4a6985e305006b2f97eaf90"
Jul 07 14:45:14.672642 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:14.672625    2570 scope.go:117] "RemoveContainer" containerID="f56aa890c82f38e7a3b76d7583ff099cd20d7e0189f78ee04866746c7fe0d356"
Jul 07 14:45:14.681737 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:14.681715    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"]
Jul 07 14:45:14.685143 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:14.685124    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f31ce-predictor-85665c44b4-nq6c7"]
Jul 07 14:45:16.242281 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:16.242242    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" path="/var/lib/kubelet/pods/480cce6f-bfb8-4409-b5bd-38c19c774e6c/volumes"
Jul 07 14:45:18.663495 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:18.663467    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:18.664085 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:18.664058    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 14:45:28.664915 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:28.664879    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 14:45:31.632293 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632260    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"]
Jul 07 14:45:31.632774 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632759    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kube-rbac-proxy"
Jul 07 14:45:31.632838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632776    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kube-rbac-proxy"
Jul 07 14:45:31.632838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632812    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container"
Jul 07 14:45:31.632838 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632820    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container"
Jul 07 14:45:31.632945 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632890    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kserve-container"
Jul 07 14:45:31.632945 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.632903    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="480cce6f-bfb8-4409-b5bd-38c19c774e6c" containerName="kube-rbac-proxy"
Jul 07 14:45:31.637831 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.637815    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.640091 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.640068    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7beb0-predictor-serving-cert\""
Jul 07 14:45:31.640909 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.640892    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-7beb0-kube-rbac-proxy-sar-config\""
Jul 07 14:45:31.646596 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.645377    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"]
Jul 07 14:45:31.670984 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.670952    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"]
Jul 07 14:45:31.671211 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.671184    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container" containerID="cri-o://91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625" gracePeriod=30
Jul 07 14:45:31.671294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.671218    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kube-rbac-proxy" containerID="cri-o://48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832" gracePeriod=30
Jul 07 14:45:31.754207 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.754182    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.754338 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.754265    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.754338 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.754295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkdjv\" (UniqueName: \"kubernetes.io/projected/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-api-access-nkdjv\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.855177 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.855121    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.855278 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.855194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkdjv\" (UniqueName: \"kubernetes.io/projected/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-api-access-nkdjv\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.855278 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:45:31.855261    2570 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-serving-cert: secret "error-404-isvc-7beb0-predictor-serving-cert" not found
Jul 07 14:45:31.855395 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.855276    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.855395 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:45:31.855322    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls podName:4e2b4d1b-83c1-4c6e-969d-3beaaaedad89 nodeName:}" failed. No retries permitted until 2026-07-07 14:45:32.355308687 +0000 UTC m=+2186.644217763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls") pod "error-404-isvc-7beb0-predictor-5754d884b4-zxsts" (UID: "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89") : secret "error-404-isvc-7beb0-predictor-serving-cert" not found
Jul 07 14:45:31.855867 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.855847    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:31.863761 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:31.863739    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkdjv\" (UniqueName: \"kubernetes.io/projected/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-api-access-nkdjv\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:32.358248 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.358219    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:32.360587 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.360564    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls\") pod \"error-404-isvc-7beb0-predictor-5754d884b4-zxsts\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") " pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:32.551285 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.551244    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:32.679247 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.679222    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"]
Jul 07 14:45:32.681663 ip-10-0-130-145 kubenswrapper[2570]: W0707 14:45:32.681638    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2b4d1b_83c1_4c6e_969d_3beaaaedad89.slice/crio-f9abfbd306f64a4664f0cb32c7f3f431924008dd4f57a472c3a946644c3c0ab1 WatchSource:0}: Error finding container f9abfbd306f64a4664f0cb32c7f3f431924008dd4f57a472c3a946644c3c0ab1: Status 404 returned error can't find the container with id f9abfbd306f64a4664f0cb32c7f3f431924008dd4f57a472c3a946644c3c0ab1
Jul 07 14:45:32.750027 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.750002    2570 generic.go:356] "Generic (PLEG): container finished" podID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerID="48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832" exitCode=2
Jul 07 14:45:32.750121 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.750066    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" event={"ID":"42b9e97d-579f-4172-be60-2e8a3b22930a","Type":"ContainerDied","Data":"48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832"}
Jul 07 14:45:32.751686 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.751663    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" event={"ID":"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89","Type":"ContainerStarted","Data":"493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d"}
Jul 07 14:45:32.751765 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:32.751718    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" event={"ID":"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89","Type":"ContainerStarted","Data":"f9abfbd306f64a4664f0cb32c7f3f431924008dd4f57a472c3a946644c3c0ab1"}
Jul 07 14:45:33.418385 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:33.418343    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused"
Jul 07 14:45:33.757886 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:33.757808    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" event={"ID":"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89","Type":"ContainerStarted","Data":"eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb"}
Jul 07 14:45:33.758292 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:33.757892    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:33.758292 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:33.757948    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:33.759280 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:33.759252    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 14:45:33.777616 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:33.777561    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podStartSLOduration=2.777550252 podStartE2EDuration="2.777550252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:45:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:45:33.775296634 +0000 UTC m=+2188.064205732" watchObservedRunningTime="2026-07-07 14:45:33.777550252 +0000 UTC m=+2188.066459349"
Jul 07 14:45:34.522816 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.522796    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:45:34.677563 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.677537    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42b9e97d-579f-4172-be60-2e8a3b22930a-proxy-tls\") pod \"42b9e97d-579f-4172-be60-2e8a3b22930a\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") "
Jul 07 14:45:34.677683 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.677583    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-rbac-proxy-sar-config\") pod \"42b9e97d-579f-4172-be60-2e8a3b22930a\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") "
Jul 07 14:45:34.677683 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.677615    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6qmd\" (UniqueName: \"kubernetes.io/projected/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-api-access-z6qmd\") pod \"42b9e97d-579f-4172-be60-2e8a3b22930a\" (UID: \"42b9e97d-579f-4172-be60-2e8a3b22930a\") "
Jul 07 14:45:34.677953 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.677918    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "42b9e97d-579f-4172-be60-2e8a3b22930a" (UID: "42b9e97d-579f-4172-be60-2e8a3b22930a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:45:34.679645 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.679623    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-api-access-z6qmd" (OuterVolumeSpecName: "kube-api-access-z6qmd") pod "42b9e97d-579f-4172-be60-2e8a3b22930a" (UID: "42b9e97d-579f-4172-be60-2e8a3b22930a"). InnerVolumeSpecName "kube-api-access-z6qmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:45:34.679829 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.679808    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42b9e97d-579f-4172-be60-2e8a3b22930a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "42b9e97d-579f-4172-be60-2e8a3b22930a" (UID: "42b9e97d-579f-4172-be60-2e8a3b22930a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:45:34.763721 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.763698    2570 generic.go:356] "Generic (PLEG): container finished" podID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerID="91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625" exitCode=0
Jul 07 14:45:34.764067 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.763747    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"
Jul 07 14:45:34.764067 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.763754    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" event={"ID":"42b9e97d-579f-4172-be60-2e8a3b22930a","Type":"ContainerDied","Data":"91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625"}
Jul 07 14:45:34.764067 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.763812    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn" event={"ID":"42b9e97d-579f-4172-be60-2e8a3b22930a","Type":"ContainerDied","Data":"85de76446760b98100ad8bede87db48780348dfb68faf69984753e6b8d5f219c"}
Jul 07 14:45:34.764067 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.763840    2570 scope.go:117] "RemoveContainer" containerID="48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832"
Jul 07 14:45:34.764294 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.764129    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 14:45:34.774883 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.774861    2570 scope.go:117] "RemoveContainer" containerID="91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625"
Jul 07 14:45:34.778423 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.778398    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42b9e97d-579f-4172-be60-2e8a3b22930a-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:45:34.778528 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.778426    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:45:34.778528 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.778441    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6qmd\" (UniqueName: \"kubernetes.io/projected/42b9e97d-579f-4172-be60-2e8a3b22930a-kube-api-access-z6qmd\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:45:34.784546 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.784528    2570 scope.go:117] "RemoveContainer" containerID="48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832"
Jul 07 14:45:34.784785 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:45:34.784768    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832\": container with ID starting with 48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832 not found: ID does not exist" containerID="48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832"
Jul 07 14:45:34.784862 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.784790    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832"} err="failed to get container status \"48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832\": rpc error: code = NotFound desc = could not find container \"48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832\": container with ID starting with 48570317036b09c730b11ffe1bf9549f9b5f4bd6f25f5302ba8633c719a74832 not found: ID does not exist"
Jul 07 14:45:34.784862 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.784805    2570 scope.go:117] "RemoveContainer" containerID="91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625"
Jul 07 14:45:34.785053 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:45:34.785008    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625\": container with ID starting with 91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625 not found: ID does not exist" containerID="91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625"
Jul 07 14:45:34.785053 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.785026    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625"} err="failed to get container status \"91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625\": rpc error: code = NotFound desc = could not find container \"91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625\": container with ID starting with 91a7fb2337a84a58623a107fbc4778beb72b471d30b7b744cfe0eef7426be625 not found: ID does not exist"
Jul 07 14:45:34.788331 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.788311    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"]
Jul 07 14:45:34.791907 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:34.791887    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-6e67f-predictor-88d98bb6d-dl8kn"]
Jul 07 14:45:36.244022 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:36.243985    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" path="/var/lib/kubelet/pods/42b9e97d-579f-4172-be60-2e8a3b22930a/volumes"
Jul 07 14:45:38.664552 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:38.664508    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 14:45:39.769046 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:39.769016    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:45:39.769533 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:39.769504    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 14:45:48.664879 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:48.664840    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused"
Jul 07 14:45:49.770530 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:49.770486    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 14:45:58.664966 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:58.664939    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 14:45:59.769458 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:45:59.769415    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 14:46:09.769786 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:46:09.769735    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused"
Jul 07 14:46:19.770542 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:46:19.770507    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:49:06.324964 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:49:06.324937    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:49:06.327457 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:49:06.325179    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:54:06.352211 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:06.352190    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:54:06.357461 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:06.357441    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:54:46.562207 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:46.562173    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"]
Jul 07 14:54:46.562670 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:46.562409    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container" containerID="cri-o://493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d" gracePeriod=30
Jul 07 14:54:46.562670 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:46.562466    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kube-rbac-proxy" containerID="cri-o://eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb" gracePeriod=30
Jul 07 14:54:47.332051 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:47.332022    2570 generic.go:356] "Generic (PLEG): container finished" podID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerID="eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb" exitCode=2
Jul 07 14:54:47.332242 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:47.332100    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" event={"ID":"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89","Type":"ContainerDied","Data":"eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb"}
Jul 07 14:54:49.107937 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.107912    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:54:49.182880 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.182807    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls\") pod \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") "
Jul 07 14:54:49.183013 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.182891    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-rbac-proxy-sar-config\") pod \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") "
Jul 07 14:54:49.183013 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.182965    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkdjv\" (UniqueName: \"kubernetes.io/projected/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-api-access-nkdjv\") pod \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\" (UID: \"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89\") "
Jul 07 14:54:49.183312 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.183288    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" (UID: "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:54:49.184808 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.184789    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" (UID: "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:54:49.184859 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.184820    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-api-access-nkdjv" (OuterVolumeSpecName: "kube-api-access-nkdjv") pod "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" (UID: "4e2b4d1b-83c1-4c6e-969d-3beaaaedad89"). InnerVolumeSpecName "kube-api-access-nkdjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:54:49.284553 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.284529    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkdjv\" (UniqueName: \"kubernetes.io/projected/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-api-access-nkdjv\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:54:49.284553 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.284553    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:54:49.284780 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.284566    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 14:54:49.341697 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.341670    2570 generic.go:356] "Generic (PLEG): container finished" podID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerID="493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d" exitCode=0
Jul 07 14:54:49.341813 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.341719    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"
Jul 07 14:54:49.341813 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.341759    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" event={"ID":"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89","Type":"ContainerDied","Data":"493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d"}
Jul 07 14:54:49.341887 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.341813    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts" event={"ID":"4e2b4d1b-83c1-4c6e-969d-3beaaaedad89","Type":"ContainerDied","Data":"f9abfbd306f64a4664f0cb32c7f3f431924008dd4f57a472c3a946644c3c0ab1"}
Jul 07 14:54:49.341887 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.341840    2570 scope.go:117] "RemoveContainer" containerID="eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb"
Jul 07 14:54:49.350458 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.350440    2570 scope.go:117] "RemoveContainer" containerID="493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d"
Jul 07 14:54:49.357522 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.357506    2570 scope.go:117] "RemoveContainer" containerID="eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb"
Jul 07 14:54:49.357742 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:54:49.357726    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb\": container with ID starting with eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb not found: ID does not exist" containerID="eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb"
Jul 07 14:54:49.357793 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.357749    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb"} err="failed to get container status \"eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb\": rpc error: code = NotFound desc = could not find container \"eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb\": container with ID starting with eda6f57cbbd6bc969c25724190344d51d898f7f9246dee97c7b31a56aec1fcbb not found: ID does not exist"
Jul 07 14:54:49.357793 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.357765    2570 scope.go:117] "RemoveContainer" containerID="493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d"
Jul 07 14:54:49.357952 ip-10-0-130-145 kubenswrapper[2570]: E0707 14:54:49.357938    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d\": container with ID starting with 493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d not found: ID does not exist" containerID="493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d"
Jul 07 14:54:49.357991 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.357956    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d"} err="failed to get container status \"493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d\": rpc error: code = NotFound desc = could not find container \"493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d\": container with ID starting with 493ed4aa903765b97835d6b5a0a177de7eeed856a8bbe7abb10d200aa457d43d not found: ID does not exist"
Jul 07 14:54:49.367561 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.367541    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"]
Jul 07 14:54:49.370974 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:49.370957    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-7beb0-predictor-5754d884b4-zxsts"]
Jul 07 14:54:50.241272 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:54:50.241238    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" path="/var/lib/kubelet/pods/4e2b4d1b-83c1-4c6e-969d-3beaaaedad89/volumes"
Jul 07 14:59:06.382172 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:59:06.382059    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 14:59:06.386184 ip-10-0-130-145 kubenswrapper[2570]: I0707 14:59:06.382992    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 15:02:30.453890 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:30.453856    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"]
Jul 07 15:02:30.454403 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:30.454069    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container" containerID="cri-o://a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79" gracePeriod=30
Jul 07 15:02:30.454403 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:30.454170    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kube-rbac-proxy" containerID="cri-o://f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e" gracePeriod=30
Jul 07 15:02:31.368860 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:31.368824    2570 generic.go:356] "Generic (PLEG): container finished" podID="fface55e-8428-4b30-aef5-14f023cb75ae" containerID="f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e" exitCode=2
Jul 07 15:02:31.368860 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:31.368865    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" event={"ID":"fface55e-8428-4b30-aef5-14f023cb75ae","Type":"ContainerDied","Data":"f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e"}
Jul 07 15:02:33.204322 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.204300    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 15:02:33.309667 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.309588    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls\") pod \"fface55e-8428-4b30-aef5-14f023cb75ae\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") "
Jul 07 15:02:33.309667 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.309656    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6vw6\" (UniqueName: \"kubernetes.io/projected/fface55e-8428-4b30-aef5-14f023cb75ae-kube-api-access-t6vw6\") pod \"fface55e-8428-4b30-aef5-14f023cb75ae\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") "
Jul 07 15:02:33.309830 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.309690    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fface55e-8428-4b30-aef5-14f023cb75ae-kube-rbac-proxy-sar-config\") pod \"fface55e-8428-4b30-aef5-14f023cb75ae\" (UID: \"fface55e-8428-4b30-aef5-14f023cb75ae\") "
Jul 07 15:02:33.310054 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.310031    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fface55e-8428-4b30-aef5-14f023cb75ae-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "fface55e-8428-4b30-aef5-14f023cb75ae" (UID: "fface55e-8428-4b30-aef5-14f023cb75ae"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 15:02:33.311618 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.311594    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fface55e-8428-4b30-aef5-14f023cb75ae" (UID: "fface55e-8428-4b30-aef5-14f023cb75ae"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:02:33.311730 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.311685    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fface55e-8428-4b30-aef5-14f023cb75ae-kube-api-access-t6vw6" (OuterVolumeSpecName: "kube-api-access-t6vw6") pod "fface55e-8428-4b30-aef5-14f023cb75ae" (UID: "fface55e-8428-4b30-aef5-14f023cb75ae"). InnerVolumeSpecName "kube-api-access-t6vw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:02:33.378761 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.378727    2570 generic.go:356] "Generic (PLEG): container finished" podID="fface55e-8428-4b30-aef5-14f023cb75ae" containerID="a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79" exitCode=0
Jul 07 15:02:33.378901 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.378768    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" event={"ID":"fface55e-8428-4b30-aef5-14f023cb75ae","Type":"ContainerDied","Data":"a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79"}
Jul 07 15:02:33.378901 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.378818    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb" event={"ID":"fface55e-8428-4b30-aef5-14f023cb75ae","Type":"ContainerDied","Data":"b572a40f6f70ffac6abca0d981ce7ea3860b11ecd9a2405dd0240fe4c97ac829"}
Jul 07 15:02:33.378901 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.378844    2570 scope.go:117] "RemoveContainer" containerID="f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e"
Jul 07 15:02:33.378901 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.378773    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"
Jul 07 15:02:33.387677 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.387658    2570 scope.go:117] "RemoveContainer" containerID="a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79"
Jul 07 15:02:33.394611 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.394598    2570 scope.go:117] "RemoveContainer" containerID="f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e"
Jul 07 15:02:33.394841 ip-10-0-130-145 kubenswrapper[2570]: E0707 15:02:33.394823    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e\": container with ID starting with f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e not found: ID does not exist" containerID="f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e"
Jul 07 15:02:33.394886 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.394849    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e"} err="failed to get container status \"f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e\": rpc error: code = NotFound desc = could not find container \"f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e\": container with ID starting with f69bcbd1faf1c2f28fa758581e4734465e3bd86b3b4c8792444eea84542cd57e not found: ID does not exist"
Jul 07 15:02:33.394886 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.394863    2570 scope.go:117] "RemoveContainer" containerID="a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79"
Jul 07 15:02:33.395077 ip-10-0-130-145 kubenswrapper[2570]: E0707 15:02:33.395060    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79\": container with ID starting with a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79 not found: ID does not exist" containerID="a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79"
Jul 07 15:02:33.395112 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.395084    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79"} err="failed to get container status \"a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79\": rpc error: code = NotFound desc = could not find container \"a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79\": container with ID starting with a6f709ba94b68458158890caef054ade86fd6b2411afce91df8fc20724fbeb79 not found: ID does not exist"
Jul 07 15:02:33.407376 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.407353    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"]
Jul 07 15:02:33.411003 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.410976    2570 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fface55e-8428-4b30-aef5-14f023cb75ae-proxy-tls\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.411003 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.411007    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6vw6\" (UniqueName: \"kubernetes.io/projected/fface55e-8428-4b30-aef5-14f023cb75ae-kube-api-access-t6vw6\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.411133 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.411018    2570 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fface55e-8428-4b30-aef5-14f023cb75ae-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-145.ec2.internal\" DevicePath \"\""
Jul 07 15:02:33.415862 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:33.415843    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-36f00-predictor-cb475dffd-22vxb"]
Jul 07 15:02:34.241820 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:02:34.241790    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" path="/var/lib/kubelet/pods/fface55e-8428-4b30-aef5-14f023cb75ae/volumes"
Jul 07 15:03:06.459694 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:06.459632    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cn8n9_7b887d71-ef48-4b43-b8d2-700d180d40ff/global-pull-secret-syncer/0.log"
Jul 07 15:03:06.632678 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:06.632653    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8hx6v_ccc24eb5-cdcf-429c-81b4-1ff5ec2edf9c/konnectivity-agent/0.log"
Jul 07 15:03:06.700579 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:06.700560    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-145.ec2.internal_68eba935587531e5b65d894a8d258574/haproxy/0.log"
Jul 07 15:03:10.883814 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:10.883789    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/alertmanager/0.log"
Jul 07 15:03:10.910117 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:10.910092    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/config-reloader/0.log"
Jul 07 15:03:10.936755 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:10.936727    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/kube-rbac-proxy-web/0.log"
Jul 07 15:03:10.965544 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:10.965493    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/kube-rbac-proxy/0.log"
Jul 07 15:03:10.996493 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:10.996475    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/kube-rbac-proxy-metric/0.log"
Jul 07 15:03:11.026348 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.026329    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/prom-label-proxy/0.log"
Jul 07 15:03:11.064823 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.064805    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6adaa101-e7dc-47be-b875-94065c3b1069/init-config-reloader/0.log"
Jul 07 15:03:11.277980 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.277936    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b756d5bf9-mm9fx_1d4d4b03-5766-440a-a2dc-e1121a59e5dd/metrics-server/0.log"
Jul 07 15:03:11.313571 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.313553    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-bthnm_cd9e31bd-2f43-439a-9e71-ae4feeb3d007/monitoring-plugin/0.log"
Jul 07 15:03:11.561126 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.561038    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-st6dx_c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52/node-exporter/0.log"
Jul 07 15:03:11.590479 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.590458    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-st6dx_c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52/kube-rbac-proxy/0.log"
Jul 07 15:03:11.622370 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:11.622352    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-st6dx_c3ae9ae2-c8f0-4173-a82d-fbf000ec8a52/init-textfile/0.log"
Jul 07 15:03:12.100321 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:12.100298    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6467bfc974-4z42n_393cd7a0-39cb-4673-907b-70a42b16368c/telemeter-client/0.log"
Jul 07 15:03:12.126474 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:12.126455    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6467bfc974-4z42n_393cd7a0-39cb-4673-907b-70a42b16368c/reload/0.log"
Jul 07 15:03:12.150267 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:12.150247    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6467bfc974-4z42n_393cd7a0-39cb-4673-907b-70a42b16368c/kube-rbac-proxy/0.log"
Jul 07 15:03:14.600986 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:14.600962    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6f67b4b668-l4kst_d6c1bd6f-733a-4137-98e8-463fcc4cd7d9/console/0.log"
Jul 07 15:03:15.464435 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464405    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"]
Jul 07 15:03:15.464834 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464817    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kube-rbac-proxy"
Jul 07 15:03:15.464882 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464836    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kube-rbac-proxy"
Jul 07 15:03:15.464882 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464850    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kube-rbac-proxy"
Jul 07 15:03:15.464882 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464857    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kube-rbac-proxy"
Jul 07 15:03:15.464882 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464870    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kube-rbac-proxy"
Jul 07 15:03:15.464882 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464875    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kube-rbac-proxy"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464887    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464892    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464899    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464904    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464911    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464916    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464964    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464971    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b9e97d-579f-4172-be60-2e8a3b22930a" containerName="kube-rbac-proxy"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464976    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kube-rbac-proxy"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464983    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kube-rbac-proxy"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464990    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e2b4d1b-83c1-4c6e-969d-3beaaaedad89" containerName="kserve-container"
Jul 07 15:03:15.465038 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.464995    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fface55e-8428-4b30-aef5-14f023cb75ae" containerName="kserve-container"
Jul 07 15:03:15.469309 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.469291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.471685 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.471665    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrhtp\"/\"openshift-service-ca.crt\""
Jul 07 15:03:15.471778 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.471728    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nrhtp\"/\"kube-root-ca.crt\""
Jul 07 15:03:15.472599 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.472577    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nrhtp\"/\"default-dockercfg-w6fgr\""
Jul 07 15:03:15.477550 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.477528    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"]
Jul 07 15:03:15.553066 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.553044    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xgp6\" (UniqueName: \"kubernetes.io/projected/5ded978d-add7-4e44-80af-7426cbd65fd3-kube-api-access-4xgp6\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.553194 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.553076    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-sys\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.553194 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.553098    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-proc\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.553313 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.553211    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-podres\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.553313 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.553248    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-lib-modules\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654119 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654096    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-podres\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654127    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-lib-modules\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654175    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xgp6\" (UniqueName: \"kubernetes.io/projected/5ded978d-add7-4e44-80af-7426cbd65fd3-kube-api-access-4xgp6\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-sys\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654221    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-proc\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654238    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-podres\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654301    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-sys\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654351    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-proc\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.654438 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.654347    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ded978d-add7-4e44-80af-7426cbd65fd3-lib-modules\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.664430 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.664410    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xgp6\" (UniqueName: \"kubernetes.io/projected/5ded978d-add7-4e44-80af-7426cbd65fd3-kube-api-access-4xgp6\") pod \"perf-node-gather-daemonset-mghss\" (UID: \"5ded978d-add7-4e44-80af-7426cbd65fd3\") " pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.779734 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.779689    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:15.907971 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:15.907876    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"]
Jul 07 15:03:15.910813 ip-10-0-130-145 kubenswrapper[2570]: W0707 15:03:15.910790    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ded978d_add7_4e44_80af_7426cbd65fd3.slice/crio-3985d02ba3a87f1918c88de472dcce1b925fd3c4fb81af01cf90e4a69832b2f8 WatchSource:0}: Error finding container 3985d02ba3a87f1918c88de472dcce1b925fd3c4fb81af01cf90e4a69832b2f8: Status 404 returned error can't find the container with id 3985d02ba3a87f1918c88de472dcce1b925fd3c4fb81af01cf90e4a69832b2f8
Jul 07 15:03:16.172730 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.172712    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-95wtb_0588c337-000a-4777-bfeb-67c419ebd711/dns/0.log"
Jul 07 15:03:16.195124 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.195110    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-95wtb_0588c337-000a-4777-bfeb-67c419ebd711/kube-rbac-proxy/0.log"
Jul 07 15:03:16.353030 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.353007    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6nzt4_32823771-b3e3-4806-a6eb-4e66616a5b48/dns-node-resolver/0.log"
Jul 07 15:03:16.562899 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.562826    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss" event={"ID":"5ded978d-add7-4e44-80af-7426cbd65fd3","Type":"ContainerStarted","Data":"624dc0fd6b8d50326f46d06a71523adda00bce0195c22129e9bfa1f2fb4e9c34"}
Jul 07 15:03:16.562899 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.562869    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss" event={"ID":"5ded978d-add7-4e44-80af-7426cbd65fd3","Type":"ContainerStarted","Data":"3985d02ba3a87f1918c88de472dcce1b925fd3c4fb81af01cf90e4a69832b2f8"}
Jul 07 15:03:16.563060 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.562929    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:16.581803 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.581764    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss" podStartSLOduration=1.581754196 podStartE2EDuration="1.581754196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:03:16.580521249 +0000 UTC m=+3250.869430346" watchObservedRunningTime="2026-07-07 15:03:16.581754196 +0000 UTC m=+3250.870663294"
Jul 07 15:03:16.932909 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:16.932887    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-47k5s_64016df9-5267-45ef-bd4f-2b13535b77c4/node-ca/0.log"
Jul 07 15:03:18.039247 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:18.039221    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74ff99db5f-jzfph_aec2ddc5-6572-4ebe-a4f8-a42897d65002/router/0.log"
Jul 07 15:03:18.476220 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:18.476198    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vshns_b7c45f9e-7791-4b7c-b005-c9dbd2fd48b5/serve-healthcheck-canary/0.log"
Jul 07 15:03:19.223992 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:19.223965    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jtnbn_ecc8e344-fafd-48aa-9d19-b5f168f0ec43/kube-rbac-proxy/0.log"
Jul 07 15:03:19.256715 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:19.256696    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jtnbn_ecc8e344-fafd-48aa-9d19-b5f168f0ec43/exporter/0.log"
Jul 07 15:03:19.298116 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:19.298097    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jtnbn_ecc8e344-fafd-48aa-9d19-b5f168f0ec43/extractor/0.log"
Jul 07 15:03:21.568576 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:21.568548    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7598d7bb4-wx9bg_b4821464-b1e8-4a05-b898-1be7fa34498d/manager/0.log"
Jul 07 15:03:21.628201 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:21.628179    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-nhzx2_72155057-2313-4063-a9f4-2bc0aded0a36/server/0.log"
Jul 07 15:03:22.576390 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:22.576367    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nrhtp/perf-node-gather-daemonset-mghss"
Jul 07 15:03:26.661416 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:26.661386    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-m7z4g_4a36602f-4dd6-4ca8-aff4-646066eaf071/kube-storage-version-migrator-operator/1.log"
Jul 07 15:03:26.661951 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:26.661936    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-m7z4g_4a36602f-4dd6-4ca8-aff4-646066eaf071/kube-storage-version-migrator-operator/0.log"
Jul 07 15:03:27.665599 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:27.665568    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2zjck_be3f27ea-f77a-4be0-b234-5607a8b9ffcc/kube-multus/0.log"
Jul 07 15:03:27.945692 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:27.945625    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/kube-multus-additional-cni-plugins/0.log"
Jul 07 15:03:27.966717 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:27.966696    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/egress-router-binary-copy/0.log"
Jul 07 15:03:27.995758 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:27.995737    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/cni-plugins/0.log"
Jul 07 15:03:28.038800 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:28.038785    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/bond-cni-plugin/0.log"
Jul 07 15:03:28.104308 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:28.104289    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/routeoverride-cni/0.log"
Jul 07 15:03:28.126970 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:28.126954    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/whereabouts-cni-bincopy/0.log"
Jul 07 15:03:28.149908 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:28.149893    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jdw9x_f3888bea-77ca-4f33-9126-f1673c37beb9/whereabouts-cni/0.log"
Jul 07 15:03:28.448408 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:28.448384    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x8nq2_015c7f85-80c7-42cf-8c77-6f22642344c1/network-metrics-daemon/0.log"
Jul 07 15:03:28.469593 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:28.469569    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x8nq2_015c7f85-80c7-42cf-8c77-6f22642344c1/kube-rbac-proxy/0.log"
Jul 07 15:03:29.560879 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.560855    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-controller/0.log"
Jul 07 15:03:29.583117 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.583090    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/0.log"
Jul 07 15:03:29.584736 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.584701    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovn-acl-logging/1.log"
Jul 07 15:03:29.607331 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.607311    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/kube-rbac-proxy-node/0.log"
Jul 07 15:03:29.632426 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.632403    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/kube-rbac-proxy-ovn-metrics/0.log"
Jul 07 15:03:29.648583 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.648563    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/northd/0.log"
Jul 07 15:03:29.669567 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.669546    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/nbdb/0.log"
Jul 07 15:03:29.701237 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.701203    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/sbdb/0.log"
Jul 07 15:03:29.743159 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:29.743128    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mns74_5979d983-b592-460c-a540-bf0049e7071e/ovnkube-controller/0.log"
Jul 07 15:03:31.027496 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:31.027470    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-v6sph_ad127895-5cdb-4412-81f3-f6ef235d8c7b/network-check-target-container/0.log"
Jul 07 15:03:32.016875 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:32.016851    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-r8m8z_bd68a3bd-c3d7-4694-bc4d-70e61be7d632/iptables-alerter/0.log"
Jul 07 15:03:32.708427 ip-10-0-130-145 kubenswrapper[2570]: I0707 15:03:32.708403    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-rf5kb_f2436106-b7a9-47b1-94ba-0b687b02fa92/tuned/0.log"