Jun 09 14:12:15.216770 ip-10-0-128-214 systemd[1]: Starting Kubernetes Kubelet... Jun 09 14:12:15.689500 ip-10-0-128-214 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 14:12:15.689500 ip-10-0-128-214 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 14:12:15.689500 ip-10-0-128-214 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 14:12:15.689500 ip-10-0-128-214 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 14:12:15.689500 ip-10-0-128-214 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 14:12:15.690601 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.690509 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 14:12:15.698391 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698365 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 14:12:15.698391 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698386 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 14:12:15.698391 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698389 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 14:12:15.698391 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698392 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 14:12:15.698391 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698395 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698411 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698423 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698426 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698429 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698432 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698434 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698437 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698439 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698442 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698445 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698448 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698450 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698453 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698455 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698458 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698461 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698464 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698466 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698469 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 14:12:15.698555 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698471 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698474 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698476 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698479 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698481 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698484 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698486 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698488 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698491 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698494 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698497 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698499 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698501 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698504 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698508 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698511 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698514 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698517 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698519 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698521 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 14:12:15.699035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698524 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698526 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698529 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698531 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698534 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698537 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698539 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698544 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698548 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698550 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698553 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698556 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698558 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698561 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698563 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698566 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698568 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698571 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698573 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 14:12:15.699731 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698576 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698578 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698581 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698583 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698586 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698589 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698593 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698597 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698600 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698603 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698606 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698609 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698612 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698615 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698620 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698623 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698626 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698629 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698632 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 14:12:15.700289 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698634 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698637 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698639 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698642 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698644 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698647 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698649 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698652 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698655 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698657 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698660 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698662 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698665 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698668 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698670 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698673 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 14:12:15.700841 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.698676 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 14:12:15.701357 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701339 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 14:12:15.701357 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701357 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701361 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701364 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701368 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701371 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701374 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701378 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701381 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701384 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701387 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701389 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701392 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701395 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701414 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701417 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701420 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701423 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701426 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701429 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701432 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 14:12:15.701426 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701435 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701437 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701440 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701443 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701446 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701448 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701450 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701453 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701455 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701459 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701461 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701466 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701469 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701472 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701475 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701478 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701481 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701484 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701487 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701489 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 14:12:15.701956 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701492 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701494 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701496 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701499 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701502 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701505 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701507 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701510 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701512 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701516 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701519 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701521 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701525 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701529 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701532 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701535 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701538 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701540 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701543 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 14:12:15.702468 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701545 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701548 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701550 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701554 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701556 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701559 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701561 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701564 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701567 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701569 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701572 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701574 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701577 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701579 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701583 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701585 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701588 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701590 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701593 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701595 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 14:12:15.702969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701597 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701601 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701605 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701609 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701612 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701615 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701618 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701622 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701625 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701628 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701630 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701633 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701636 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701638 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701641 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701643 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701646 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701648 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.701651 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 14:12:15.703490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702051 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702056 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702059 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702062 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702064 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702067 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702070 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702072 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702077 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702080 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702083 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702087 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702089 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702092 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702095 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702097 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702100 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702103 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702106 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 14:12:15.703958 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702108 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702111 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702114 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702116 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702119 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702121 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702124 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702126 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702128 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702131 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702134 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702136 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702138 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702141 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702143 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702146 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702149 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702152 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702155 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702163 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 14:12:15.704487 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702166 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702168 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702170 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702173 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702175 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702178 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702180 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702182 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702185 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702188 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702191 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702193 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702196 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702199 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702201 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702204 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702207 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702209 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702212 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702214 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 14:12:15.704986 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702216 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702219 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702221 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702223 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702226 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702228 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702231 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702233 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702235 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702238 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702240 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702243 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702247 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702250 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702253 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702256 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702260 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702262 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702265 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702267 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 14:12:15.705490 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702269 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702272 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702275 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702278 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702280 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702283 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702286 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702288 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702291 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702293 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702295 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702298 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702300 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702303 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702305 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702308 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702310 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702312 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702315 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.702317 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 14:12:15.705976 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702415 2568 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702425 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702432 2568 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702436 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702440 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702443 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702447 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702452 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702455 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702457 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702461 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702464 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702467 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702470 2568 flags.go:64] FLAG: --cgroup-root="" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702473 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702477 2568 flags.go:64] FLAG: --client-ca-file="" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702481 2568 flags.go:64] FLAG: --cloud-provider="external" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702485 2568 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702490 2568 flags.go:64] FLAG: --cluster-domain="" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702493 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702496 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702499 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702502 2568 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702506 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 14:12:15.706503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702509 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702512 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702514 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702517 2568 flags.go:64] FLAG: --contention-profiling="false" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702520 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702523 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702526 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702528 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702533 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702535 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702538 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702541 2568 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702544 2568 flags.go:64] FLAG: --enable-server="true" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702546 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702551 2568 flags.go:64] FLAG: --event-burst="100" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702553 2568 flags.go:64] FLAG: --event-qps="50" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702556 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702559 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702562 2568 flags.go:64] FLAG: --eviction-hard="" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702565 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702568 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702571 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702574 2568 flags.go:64] FLAG: --eviction-soft="" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702577 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702580 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 14:12:15.707072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702584 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702587 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702589 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702592 2568 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702595 2568 flags.go:64] FLAG: --feature-gates="" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702598 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702601 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702604 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702607 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702610 2568 flags.go:64] FLAG: --healthz-port="10248" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702613 2568 flags.go:64] FLAG: --help="false" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702615 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702618 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702621 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702624 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702627 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702630 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702633 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702636 2568 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702638 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702641 2568 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702644 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702647 2568 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702650 2568 flags.go:64] FLAG: --kube-reserved="" Jun 09 14:12:15.707844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702652 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702655 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702658 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702661 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702664 2568 flags.go:64] FLAG: --lock-file="" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702667 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702669 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702672 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702678 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702681 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702684 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702687 2568 flags.go:64] FLAG: --logging-format="text" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702690 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702693 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702695 2568 flags.go:64] FLAG: --manifest-url="" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702698 2568 flags.go:64] FLAG: --manifest-url-header="" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702702 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702705 2568 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702709 2568 flags.go:64] FLAG: --max-pods="110" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702712 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702715 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702717 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702720 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702723 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702726 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 14:12:15.708448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702728 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702736 2568 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702739 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702741 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702744 2568 flags.go:64] FLAG: --pod-cidr="" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702747 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702752 2568 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702755 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702759 2568 flags.go:64] FLAG: --pods-per-core="0" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702761 2568 flags.go:64] FLAG: --port="10250" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702764 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702767 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04dd4f0af47437d76" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702770 2568 flags.go:64] FLAG: --qos-reserved="" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702773 2568 flags.go:64] FLAG: --read-only-port="10255" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702776 2568 flags.go:64] FLAG: --register-node="true" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702779 2568 flags.go:64] FLAG: --register-with-taints="" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702782 2568 flags.go:64] FLAG: --registry-burst="10" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702786 2568 flags.go:64] FLAG: --registry-qps="5" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702789 2568 flags.go:64] FLAG: --reserved-cpus="" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702792 2568 flags.go:64] FLAG: --reserved-memory="" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702796 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702798 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702801 2568 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702804 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702807 2568 flags.go:64] FLAG: --runonce="false" Jun 09 14:12:15.709068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702809 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702812 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702815 2568 flags.go:64] FLAG: --seccomp-default="false" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702817 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702820 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702823 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702825 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702828 2568 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702831 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702834 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702836 2568 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702839 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702842 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702844 2568 flags.go:64] FLAG: --system-cgroups="" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702847 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702855 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702858 2568 flags.go:64] FLAG: --tls-cert-file="" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702860 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702866 2568 flags.go:64] FLAG: --tls-min-version="" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702868 2568 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702871 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702874 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702879 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702882 2568 flags.go:64] FLAG: --v="2" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702886 2568 flags.go:64] FLAG: --version="false" Jun 09 14:12:15.710109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702890 2568 flags.go:64] FLAG: --vmodule="" Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702894 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.702897 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703018 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703022 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703025 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703028 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703030 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703033 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703035 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703038 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703040 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703043 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703046 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703048 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703051 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703053 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703056 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703058 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703061 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 14:12:15.711244 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703063 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703066 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703069 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703072 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703075 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703077 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703080 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703083 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703085 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703089 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703092 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703094 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703096 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703100 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703102 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703106 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703109 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703112 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703115 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 14:12:15.711883 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703117 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703121 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703125 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703128 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703131 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703134 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703137 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703139 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703142 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703145 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703147 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703150 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703152 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703156 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703158 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703163 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703165 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703168 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703171 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 14:12:15.712355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703173 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703176 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703178 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703182 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703184 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703187 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703190 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703193 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703196 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703199 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703201 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703204 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703206 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703209 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703211 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703214 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703217 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703221 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703223 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703226 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 14:12:15.713193 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703230 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703232 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703235 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703238 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703240 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703243 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703246 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703249 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703252 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703255 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703258 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703260 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703263 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703266 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703268 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703272 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703274 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703277 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703280 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703282 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 14:12:15.714035 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703285 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703288 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703291 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.703294 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.703299 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.711465 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.711491 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.712359 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.712370 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712511 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712521 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712526 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712531 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712536 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 14:12:15.714715 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712540 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712544 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712549 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712554 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712559 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712566 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712572 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712577 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712581 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712586 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712590 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712595 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712599 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712603 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712608 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712612 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712617 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712622 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712626 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 14:12:15.715300 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712630 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712634 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712639 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712643 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712648 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712654 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712658 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712663 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712667 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712671 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712675 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712680 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712684 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712688 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712692 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712696 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712710 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712714 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712718 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712722 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 14:12:15.716100 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712727 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712731 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712735 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712739 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712743 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712747 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712752 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712756 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712760 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712767 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712772 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712777 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712784 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712790 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712794 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712799 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712805 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712810 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712816 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 14:12:15.716642 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712820 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712824 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712829 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712833 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712839 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712843 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712848 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712852 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712857 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712861 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712865 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712870 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712874 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712878 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712882 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712886 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712891 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712895 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712899 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712904 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 14:12:15.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712908 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712911 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712916 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712921 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712925 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712930 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712934 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712938 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712942 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712947 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712952 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712956 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712962 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712966 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712971 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 14:12:15.717783 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.712975 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.712983 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713168 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713176 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713181 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713185 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713189 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713193 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713197 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713201 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713205 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713209 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713213 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713217 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713221 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 14:12:15.718243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713225 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713230 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713234 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713238 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713243 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713247 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713251 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713255 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713259 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713264 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713268 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713272 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713277 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713282 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713287 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713292 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713297 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713302 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713309 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713313 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 14:12:15.718693 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713318 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713322 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713326 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713330 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713334 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713341 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713346 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713351 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713355 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713359 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713363 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713368 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713372 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713376 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713380 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713384 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713389 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713393 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713416 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 14:12:15.719243 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713421 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713425 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713429 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713433 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713437 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713442 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713447 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713451 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713455 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713460 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713465 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713469 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713473 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713477 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713481 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713485 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713490 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713493 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713498 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713501 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 14:12:15.719748 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713505 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713510 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713514 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713518 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713522 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713526 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713531 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713535 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713539 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713544 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713548 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713553 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713557 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713561 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713566 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713570 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713574 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713580 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713586 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 14:12:15.720240 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713591 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713596 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713601 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713606 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713611 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713615 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713619 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:15.713623 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.713631 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.713831 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.716737 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.718089 2568 server.go:1019] "Starting client certificate rotation" Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.718200 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 14:12:15.720947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.718242 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 14:12:15.721433 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.721152 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 14:12:15.724627 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.724603 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 14:12:15.735372 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.735347 2568 log.go:25] "Validated CRI v1 runtime API" Jun 09 14:12:15.738020 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.738002 2568 log.go:25] "Validated CRI v1 image API" Jun 09 14:12:15.740598 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.740583 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 14:12:15.743730 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.743699 2568 fs.go:135] Filesystem UUIDs: map[0469d227-cfdd-46b7-b404-282b3f2f22a6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c74d2f44-778b-4de5-9b3e-a9b0645ff8d5:/dev/nvme0n1p3] Jun 09 14:12:15.743814 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.743729 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 14:12:15.749152 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.749033 2568 manager.go:216] Machine: {Timestamp:2026-06-09 14:12:15.748503763 +0000 UTC m=+0.395250446 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100610 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f35670265d5dda055f0823e7c4458 SystemUUID:ec2f3567-0265-d5dd-a055-f0823e7c4458 BootID:8fa796a3-6ac1-4e00-9492-b38c3a9d853b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2f:94:02:d9:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2f:94:02:d9:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:f3:90:e1:4b:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 14:12:15.749152 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.749141 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 14:12:15.749315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.749221 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 14:12:15.752849 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.752820 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 14:12:15.753013 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.752853 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-214.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 14:12:15.754092 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.754079 2568 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 14:12:15.754155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.754098 2568 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 14:12:15.754155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.754110 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 14:12:15.754155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.754126 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 14:12:15.755021 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.755007 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 14:12:15.757566 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.757548 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 14:12:15.757979 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.757965 2568 state_mem.go:36] "Initialized new in-memory state store" Jun 09 14:12:15.758118 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758107 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 14:12:15.758173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758146 2568 kubelet.go:486] "Attempting to sync node with API server" Jun 09 14:12:15.758173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758161 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 14:12:15.758276 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758176 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 14:12:15.758276 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758190 2568 kubelet.go:398] "Adding apiserver pod source" Jun 09 14:12:15.758276 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758202 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 14:12:15.758658 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758644 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 14:12:15.758738 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.758669 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 14:12:15.765144 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.765117 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 14:12:15.770676 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.770658 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 14:12:15.770749 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.770690 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 14:12:15.771389 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771373 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 14:12:15.771474 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771430 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 14:12:15.771474 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771443 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 14:12:15.771474 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771461 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771479 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771490 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771504 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771512 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.771512 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-214.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.771515 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 14:12:15.771559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771525 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 14:12:15.771742 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771600 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 14:12:15.771742 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771611 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 14:12:15.771742 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.771618 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 14:12:15.774124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.774111 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 14:12:15.774124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.774124 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 14:12:15.775990 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.775969 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-214.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 14:12:15.777586 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.777573 2568 server.go:1268] "Started kubelet" Jun 09 14:12:15.777746 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.777718 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 14:12:15.777847 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.777657 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 14:12:15.777924 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.777890 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 14:12:15.778622 ip-10-0-128-214 systemd[1]: Started Kubernetes Kubelet. Jun 09 14:12:15.779309 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.779222 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2g6kh" Jun 09 14:12:15.779667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.779617 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 14:12:15.780150 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.780134 2568 server.go:310] "Adding debug handlers to kubelet server" Jun 09 14:12:15.784693 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.784667 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 14:12:15.785357 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.785343 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 14:12:15.785506 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.785343 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 14:12:15.785700 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.785667 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:15.786470 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786455 2568 factory.go:55] Registering systemd factory Jun 09 14:12:15.786541 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786474 2568 factory.go:223] Registration of the systemd container factory successfully Jun 09 14:12:15.786541 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786533 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 14:12:15.786624 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786535 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 14:12:15.786624 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786560 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 14:12:15.786725 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786711 2568 reconstruct.go:97] "Volume reconstruction finished" Jun 09 14:12:15.786725 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786725 2568 reconciler.go:29] "Reconciler: start to sync state" Jun 09 14:12:15.786829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786763 2568 factory.go:153] Registering CRI-O factory Jun 09 14:12:15.786829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786771 2568 factory.go:223] Registration of the crio container factory successfully Jun 09 14:12:15.786829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786828 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 14:12:15.786921 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786885 2568 factory.go:103] Registering Raw factory Jun 09 14:12:15.786921 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786901 2568 manager.go:1196] Started watching for new ooms in manager Jun 09 14:12:15.787013 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.786994 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2g6kh" Jun 09 14:12:15.787724 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.787712 2568 manager.go:349] Starting recovery of all containers Jun 09 14:12:15.797798 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.797344 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:15.801429 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.801378 2568 manager.go:354] Recovery completed Jun 09 14:12:15.803139 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.803120 2568 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 09 14:12:15.804982 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.804963 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-214.ec2.internal\" not found" node="ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.806306 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.806293 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.808717 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.808569 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.808717 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.808608 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.808717 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.808619 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.809237 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.809221 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 14:12:15.809237 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.809234 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 14:12:15.809362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.809251 2568 state_mem.go:36] "Initialized new in-memory state store" Jun 09 14:12:15.809362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.809237 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.810046 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.810033 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.810085 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.810060 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.810085 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.810076 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.810788 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.810776 2568 policy_none.go:49] "None policy: Start" Jun 09 14:12:15.810839 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.810794 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 14:12:15.810839 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.810804 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 14:12:15.812357 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.812346 2568 policy_none.go:47] "Start" Jun 09 14:12:15.856143 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856123 2568 manager.go:337] "Starting Device Plugin manager" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.856160 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856171 2568 server.go:87] "Starting device plugin registration server" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856596 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856618 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856873 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856947 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.856956 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.857381 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.857453 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.869378 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.870703 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.870740 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.870763 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.870802 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 14:12:15.874091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.873310 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:15.957853 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.957758 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.958993 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.958979 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.959062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.959008 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.959062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.959018 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.959062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.959039 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.968156 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.968137 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.968273 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.968162 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-214.ec2.internal\": node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:15.971749 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.971720 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal"] Jun 09 14:12:15.971805 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.971795 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.973336 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.973318 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.973461 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.973351 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.973461 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.973365 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.975338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.975325 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.975478 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.975463 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.975535 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.975500 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.976031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.976016 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.976109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.976041 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.976109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.976057 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.976109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.976083 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.976109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.976107 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.976282 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.976120 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.977809 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.977793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.977904 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.977823 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 14:12:15.978474 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.978457 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientMemory" Jun 09 14:12:15.978568 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.978480 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 14:12:15.978568 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.978493 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeHasSufficientPID" Jun 09 14:12:15.981573 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.981556 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:15.988213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.988197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a72335425bac515a27766cdcfbe29358-config\") pod \"kube-apiserver-proxy-ip-10-0-128-214.ec2.internal\" (UID: \"a72335425bac515a27766cdcfbe29358\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.988288 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.988221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ad19473c8d2169042690ba1f43febb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal\" (UID: \"3ad19473c8d2169042690ba1f43febb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.988288 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:15.988246 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ad19473c8d2169042690ba1f43febb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal\" (UID: \"3ad19473c8d2169042690ba1f43febb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.993991 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.993975 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-214.ec2.internal\" not found" node="ip-10-0-128-214.ec2.internal" Jun 09 14:12:15.997899 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:15.997882 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-214.ec2.internal\" not found" node="ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.082388 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.082358 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:16.088529 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.088512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a72335425bac515a27766cdcfbe29358-config\") pod \"kube-apiserver-proxy-ip-10-0-128-214.ec2.internal\" (UID: \"a72335425bac515a27766cdcfbe29358\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.088619 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.088536 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ad19473c8d2169042690ba1f43febb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal\" (UID: \"3ad19473c8d2169042690ba1f43febb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.088619 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.088553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ad19473c8d2169042690ba1f43febb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal\" (UID: \"3ad19473c8d2169042690ba1f43febb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.088619 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.088579 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ad19473c8d2169042690ba1f43febb8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal\" (UID: \"3ad19473c8d2169042690ba1f43febb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.088722 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.088622 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a72335425bac515a27766cdcfbe29358-config\") pod \"kube-apiserver-proxy-ip-10-0-128-214.ec2.internal\" (UID: \"a72335425bac515a27766cdcfbe29358\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.088722 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.088640 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ad19473c8d2169042690ba1f43febb8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal\" (UID: \"3ad19473c8d2169042690ba1f43febb8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.182814 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.182780 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:16.283647 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.283619 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:16.299155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.299129 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.302638 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.302613 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.384384 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.384341 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:16.484967 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.484934 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:16.585527 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.585440 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-214.ec2.internal\" not found" Jun 09 14:12:16.651231 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.651198 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:16.686014 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.685970 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.700295 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.700267 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 14:12:16.702543 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.702525 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" Jun 09 14:12:16.710722 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.710700 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 14:12:16.717776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.717760 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 14:12:16.717899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.717875 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 14:12:16.718018 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.717909 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 14:12:16.718018 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.717922 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 14:12:16.755682 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.755641 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:16.758565 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.758541 2568 apiserver.go:52] "Watching apiserver" Jun 09 14:12:16.765480 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.765458 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 14:12:16.766719 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.766692 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal","openshift-multus/network-metrics-daemon-hsxpw","openshift-ovn-kubernetes/ovnkube-node-2wmnf","kube-system/global-pull-secret-syncer-bcb8m","openshift-cluster-node-tuning-operator/tuned-7dc9x","openshift-dns/node-resolver-rk89k","openshift-image-registry/node-ca-vvsn4","openshift-multus/multus-additional-cni-plugins-mkqk7","openshift-multus/multus-k7ngj","openshift-network-diagnostics/network-check-target-lsq9c","openshift-network-operator/iptables-alerter-4zh7k","kube-system/konnectivity-agent-7nrcw"] Jun 09 14:12:16.769048 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.769029 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.770745 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.770719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.771388 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.771370 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 14:12:16.771504 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.771493 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 14:12:16.771570 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.771512 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.771790 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.771775 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 14:12:16.771874 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.771825 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.771874 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.771838 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t4hr2\"" Jun 09 14:12:16.772851 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.772819 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 14:12:16.772953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.772895 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.773314 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.773293 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lstl9\"" Jun 09 14:12:16.773432 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.773367 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.774017 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.773997 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.775022 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.775005 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:16.775110 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.775082 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:16.775172 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.775131 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9rdsg\"" Jun 09 14:12:16.775388 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.775372 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.775469 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.775388 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.775469 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.775379 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 14:12:16.776689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.776671 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:16.776776 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.776733 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:16.778219 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.778198 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.778335 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.778256 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:16.780470 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.780447 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.783718 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.782526 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.783718 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.782847 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m4rmr\"" Jun 09 14:12:16.783718 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.783057 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.783967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.783949 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.786017 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.785995 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 14:12:16.786123 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.786004 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.786326 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.786305 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.786703 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.786683 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.786908 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.786790 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2nwvm\"" Jun 09 14:12:16.788359 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.788343 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.788902 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.788876 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2h9d7\"" Jun 09 14:12:16.789038 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.789016 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.789162 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.789074 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.789642 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.789624 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 14:12:16.790325 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790309 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.790493 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790446 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 14:07:15 +0000 UTC" deadline="2027-11-15 09:14:44.080049389 +0000 UTC" Jun 09 14:12:16.790493 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790491 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12571h2m27.289562465s" Jun 09 14:12:16.790493 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790499 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 14:12:16.790750 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790721 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 14:12:16.790825 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790802 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5dslj\"" Jun 09 14:12:16.790935 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790916 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 14:12:16.791027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790944 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 14:12:16.791027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790925 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 14:12:16.791027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.790921 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 14:12:16.792097 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8da21813-284f-4169-b06c-dd9b636cdc8f-hosts-file\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.792173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792104 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-ovnkube-script-lib\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-systemd-units\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792157 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-os-release\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.792267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.792267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-etc-selinux\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.792267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792244 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.792416 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792269 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-host\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.792416 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792296 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 14:12:16.792416 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792314 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b5fvh\"" Jun 09 14:12:16.792416 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792293 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-kubelet\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792416 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792356 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:16.792416 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792383 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysconfig\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792427 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-node-log\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4f9f\" (UniqueName: \"kubernetes.io/projected/8393f4ed-5c8d-4685-8bc0-87d143940155-kube-api-access-l4f9f\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-socket-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdbc7\" (UniqueName: \"kubernetes.io/projected/9f56ab11-bfb5-416f-99e7-0b29e8260701-kube-api-access-cdbc7\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-var-lib-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-etc-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792587 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqm2q\" (UniqueName: \"kubernetes.io/projected/13916a76-8368-4bc7-b6d0-55615e3aab9e-kube-api-access-nqm2q\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysctl-d\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792631 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-run-netns\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-ovnkube-config\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-cnibin\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.792688 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792686 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysctl-conf\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a18c19c3-e11d-4191-81a4-73707986bee7-host\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792745 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-slash\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-run-ovn-kubernetes\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792797 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8393f4ed-5c8d-4685-8bc0-87d143940155-ovn-node-metrics-cert\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792819 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5rm7\" (UniqueName: \"kubernetes.io/projected/2f6f3949-59cd-4c78-8709-b027e61dea18-kube-api-access-k5rm7\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792842 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-registration-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792858 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45354e28-54ab-460d-981a-86b17ed5cce9-tmp\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792908 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8da21813-284f-4169-b06c-dd9b636cdc8f-tmp-dir\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792925 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-sys-fs\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-kubernetes\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbj9n\" (UniqueName: \"kubernetes.io/projected/8da21813-284f-4169-b06c-dd9b636cdc8f-kube-api-access-hbj9n\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.792980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bthhc\" (UniqueName: \"kubernetes.io/projected/a18c19c3-e11d-4191-81a4-73707986bee7-kube-api-access-bthhc\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793004 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-systemd\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793035 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-log-socket\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.793329 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793071 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-env-overrides\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-iptables-alerter-script\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793120 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-lib-modules\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a18c19c3-e11d-4191-81a4-73707986bee7-serviceca\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2g2\" (UniqueName: \"kubernetes.io/projected/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-kube-api-access-pf2g2\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a611bc1-9997-42fa-88ae-2feb6105e7bd-kubelet-config\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793187 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a611bc1-9997-42fa-88ae-2feb6105e7bd-dbus\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793283 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x67hh\" (UniqueName: \"kubernetes.io/projected/45354e28-54ab-460d-981a-86b17ed5cce9-kube-api-access-x67hh\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793315 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-ovn\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793341 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-system-cni-dir\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-cni-binary-copy\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793384 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-systemd\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793423 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-run\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-sys\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793453 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-cni-bin\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793468 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-host-slash\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.794146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793483 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-cni-netd\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-device-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-var-lib-kubelet\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/45354e28-54ab-460d-981a-86b17ed5cce9-etc-tuned\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793570 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793588 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-modprobe-d\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.793615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.794678 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dq4nj\"" Jun 09 14:12:16.794876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.794786 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 14:12:16.795248 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.794951 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 14:12:16.797768 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.797747 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 14:12:16.819027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.819000 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bfkwz" Jun 09 14:12:16.826613 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.826594 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bfkwz" Jun 09 14:12:16.885846 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:16.885799 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad19473c8d2169042690ba1f43febb8.slice/crio-fbd1df25cab61201f54d78276228b7946af8fa83b1c793dbda0ee1c9b7319020 WatchSource:0}: Error finding container fbd1df25cab61201f54d78276228b7946af8fa83b1c793dbda0ee1c9b7319020: Status 404 returned error can't find the container with id fbd1df25cab61201f54d78276228b7946af8fa83b1c793dbda0ee1c9b7319020 Jun 09 14:12:16.886132 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:16.886116 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72335425bac515a27766cdcfbe29358.slice/crio-dd34b5d2fa6883ab2846983967f62c7d6225da0edbf78da246845f9da216c55a WatchSource:0}: Error finding container dd34b5d2fa6883ab2846983967f62c7d6225da0edbf78da246845f9da216c55a: Status 404 returned error can't find the container with id dd34b5d2fa6883ab2846983967f62c7d6225da0edbf78da246845f9da216c55a Jun 09 14:12:16.887636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.887617 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 14:12:16.890298 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.890283 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 14:12:16.893870 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.893832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a611bc1-9997-42fa-88ae-2feb6105e7bd-dbus\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.893966 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.893880 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-multus-certs\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.893966 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.893913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x67hh\" (UniqueName: \"kubernetes.io/projected/45354e28-54ab-460d-981a-86b17ed5cce9-kube-api-access-x67hh\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.893966 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.893941 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-ovn\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.894135 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.893965 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-system-cni-dir\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.894135 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894013 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4a611bc1-9997-42fa-88ae-2feb6105e7bd-dbus\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894125 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-cni-binary-copy\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894150 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-system-cni-dir\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894161 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5215ace-4db9-42de-ac5d-a0a68738cbbb-agent-certs\") pod \"konnectivity-agent-7nrcw\" (UID: \"e5215ace-4db9-42de-ac5d-a0a68738cbbb\") " pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-systemd\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894185 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-ovn\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-run\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894221 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-sys\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-cni-bin\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-host-slash\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-cni-netd\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-device-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-hostroot\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894335 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-var-lib-kubelet\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894397 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-host-slash\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894423 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-device-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894458 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-sys\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894457 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/45354e28-54ab-460d-981a-86b17ed5cce9-etc-tuned\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894501 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-kubelet\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894526 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-cni-netd\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894458 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-systemd\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894546 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-modprobe-d\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894569 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.894602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894572 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-cni-bin\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894612 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-run\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-cni-binary-copy\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-socket-dir-parent\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-conf-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8da21813-284f-4169-b06c-dd9b636cdc8f-hosts-file\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894761 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-ovnkube-script-lib\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894775 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-cni-bin\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7829\" (UniqueName: \"kubernetes.io/projected/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-kube-api-access-j7829\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-systemd-units\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-os-release\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894890 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-etc-selinux\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-cni-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-k8s-cni-cncf-io\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.895338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895022 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-host\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.894946 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-modprobe-d\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895065 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-cni-binary-copy\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-kubelet\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895109 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895155 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895199 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8da21813-284f-4169-b06c-dd9b636cdc8f-hosts-file\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-cni-multus\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895229 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-etc-selinux\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-os-release\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895278 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-systemd-units\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895356 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-host\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895394 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-kubelet\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.896304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895234 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-daemon-config\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysconfig\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-node-log\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4f9f\" (UniqueName: \"kubernetes.io/projected/8393f4ed-5c8d-4685-8bc0-87d143940155-kube-api-access-l4f9f\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-socket-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895563 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895566 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895589 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdbc7\" (UniqueName: \"kubernetes.io/projected/9f56ab11-bfb5-416f-99e7-0b29e8260701-kube-api-access-cdbc7\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-netns\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-var-lib-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-etc-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895736 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2f6f3949-59cd-4c78-8709-b027e61dea18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqm2q\" (UniqueName: \"kubernetes.io/projected/13916a76-8368-4bc7-b6d0-55615e3aab9e-kube-api-access-nqm2q\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-system-cni-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895813 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-node-log\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-etc-kubernetes\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.897132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysctl-d\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895872 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-run-netns\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.895897 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895887 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-ovnkube-script-lib\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.895959 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:12:17.395939369 +0000 UTC m=+2.042686083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895989 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysconfig\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895999 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-etc-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896031 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-socket-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.895895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-ovnkube-config\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896057 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysctl-d\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-cnibin\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896076 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-var-lib-openvswitch\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896208 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-run-netns\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysctl-conf\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896258 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2f6f3949-59cd-4c78-8709-b027e61dea18-cnibin\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a18c19c3-e11d-4191-81a4-73707986bee7-host\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896330 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-ovnkube-config\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.897906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896344 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a18c19c3-e11d-4191-81a4-73707986bee7-host\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896378 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-sysctl-conf\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896383 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-slash\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896438 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-run-ovn-kubernetes\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896441 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-slash\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896476 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-var-lib-kubelet\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896491 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-host-run-ovn-kubernetes\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896522 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8393f4ed-5c8d-4685-8bc0-87d143940155-ovn-node-metrics-cert\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896574 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5rm7\" (UniqueName: \"kubernetes.io/projected/2f6f3949-59cd-4c78-8709-b027e61dea18-kube-api-access-k5rm7\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-registration-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45354e28-54ab-460d-981a-86b17ed5cce9-tmp\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8da21813-284f-4169-b06c-dd9b636cdc8f-tmp-dir\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-sys-fs\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-cnibin\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896857 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-registration-dir\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-os-release\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.898694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5215ace-4db9-42de-ac5d-a0a68738cbbb-konnectivity-ca\") pod \"konnectivity-agent-7nrcw\" (UID: \"e5215ace-4db9-42de-ac5d-a0a68738cbbb\") " pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-kubernetes\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.896933 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbj9n\" (UniqueName: \"kubernetes.io/projected/8da21813-284f-4169-b06c-dd9b636cdc8f-kube-api-access-hbj9n\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.896970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bthhc\" (UniqueName: \"kubernetes.io/projected/a18c19c3-e11d-4191-81a4-73707986bee7-kube-api-access-bthhc\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.896994 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:12:17.396975393 +0000 UTC m=+2.043722078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-systemd\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-log-socket\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897068 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-env-overrides\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897093 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8da21813-284f-4169-b06c-dd9b636cdc8f-tmp-dir\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897117 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9f56ab11-bfb5-416f-99e7-0b29e8260701-sys-fs\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897095 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-iptables-alerter-script\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-lib-modules\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897180 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-log-socket\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a18c19c3-e11d-4191-81a4-73707986bee7-serviceca\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897208 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2g2\" (UniqueName: \"kubernetes.io/projected/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-kube-api-access-pf2g2\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897213 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8393f4ed-5c8d-4685-8bc0-87d143940155-run-systemd\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a611bc1-9997-42fa-88ae-2feb6105e7bd-kubelet-config\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897284 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-etc-kubernetes\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897479 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4a611bc1-9997-42fa-88ae-2feb6105e7bd-kubelet-config\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897609 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/45354e28-54ab-460d-981a-86b17ed5cce9-lib-modules\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897634 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-iptables-alerter-script\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897674 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8393f4ed-5c8d-4685-8bc0-87d143940155-env-overrides\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.897697 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a18c19c3-e11d-4191-81a4-73707986bee7-serviceca\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.899272 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/45354e28-54ab-460d-981a-86b17ed5cce9-etc-tuned\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.899342 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8393f4ed-5c8d-4685-8bc0-87d143940155-ovn-node-metrics-cert\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.899899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.899598 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45354e28-54ab-460d-981a-86b17ed5cce9-tmp\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.902813 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.902686 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:16.902813 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.902708 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:16.902813 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.902721 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:16.902813 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.902731 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x67hh\" (UniqueName: \"kubernetes.io/projected/45354e28-54ab-460d-981a-86b17ed5cce9-kube-api-access-x67hh\") pod \"tuned-7dc9x\" (UID: \"45354e28-54ab-460d-981a-86b17ed5cce9\") " pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:16.902813 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:16.902778 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:17.402759519 +0000 UTC m=+2.049506210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:16.904241 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.904215 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqm2q\" (UniqueName: \"kubernetes.io/projected/13916a76-8368-4bc7-b6d0-55615e3aab9e-kube-api-access-nqm2q\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:16.904509 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.904492 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5rm7\" (UniqueName: \"kubernetes.io/projected/2f6f3949-59cd-4c78-8709-b027e61dea18-kube-api-access-k5rm7\") pod \"multus-additional-cni-plugins-mkqk7\" (UID: \"2f6f3949-59cd-4c78-8709-b027e61dea18\") " pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:16.904509 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.904496 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdbc7\" (UniqueName: \"kubernetes.io/projected/9f56ab11-bfb5-416f-99e7-0b29e8260701-kube-api-access-cdbc7\") pod \"aws-ebs-csi-driver-node-rkdmj\" (UID: \"9f56ab11-bfb5-416f-99e7-0b29e8260701\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:16.905470 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.905444 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4f9f\" (UniqueName: \"kubernetes.io/projected/8393f4ed-5c8d-4685-8bc0-87d143940155-kube-api-access-l4f9f\") pod \"ovnkube-node-2wmnf\" (UID: \"8393f4ed-5c8d-4685-8bc0-87d143940155\") " pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:16.905582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.905540 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bthhc\" (UniqueName: \"kubernetes.io/projected/a18c19c3-e11d-4191-81a4-73707986bee7-kube-api-access-bthhc\") pod \"node-ca-vvsn4\" (UID: \"a18c19c3-e11d-4191-81a4-73707986bee7\") " pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:16.905920 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.905903 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2g2\" (UniqueName: \"kubernetes.io/projected/8d454a83-cb18-4aa5-82b7-73fa3cfed11c-kube-api-access-pf2g2\") pod \"iptables-alerter-4zh7k\" (UID: \"8d454a83-cb18-4aa5-82b7-73fa3cfed11c\") " pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:16.906696 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.906674 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbj9n\" (UniqueName: \"kubernetes.io/projected/8da21813-284f-4169-b06c-dd9b636cdc8f-kube-api-access-hbj9n\") pod \"node-resolver-rk89k\" (UID: \"8da21813-284f-4169-b06c-dd9b636cdc8f\") " pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:16.998370 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-kubelet\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998370 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-cni-binary-copy\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998388 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-socket-dir-parent\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-conf-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-cni-bin\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998480 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7829\" (UniqueName: \"kubernetes.io/projected/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-kube-api-access-j7829\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-cni-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998513 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-cni-bin\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998519 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-socket-dir-parent\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998514 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-conf-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998542 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-k8s-cni-cncf-io\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998577 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-k8s-cni-cncf-io\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998477 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-kubelet\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.998597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998590 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-cni-multus\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998611 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-var-lib-cni-multus\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-daemon-config\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-netns\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998708 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-cni-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-system-cni-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998758 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-system-cni-dir\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998766 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-etc-kubernetes\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998797 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-netns\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998835 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-etc-kubernetes\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-cnibin\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-os-release\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5215ace-4db9-42de-ac5d-a0a68738cbbb-konnectivity-ca\") pod \"konnectivity-agent-7nrcw\" (UID: \"e5215ace-4db9-42de-ac5d-a0a68738cbbb\") " pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-multus-certs\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998963 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-cnibin\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.998977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5215ace-4db9-42de-ac5d-a0a68738cbbb-agent-certs\") pod \"konnectivity-agent-7nrcw\" (UID: \"e5215ace-4db9-42de-ac5d-a0a68738cbbb\") " pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999010 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-hostroot\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999036 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-os-release\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999054 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-cni-binary-copy\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999096 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-host-run-multus-certs\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999769 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999100 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-hostroot\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999769 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999171 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-multus-daemon-config\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:16.999769 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:16.999433 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e5215ace-4db9-42de-ac5d-a0a68738cbbb-konnectivity-ca\") pod \"konnectivity-agent-7nrcw\" (UID: \"e5215ace-4db9-42de-ac5d-a0a68738cbbb\") " pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:17.001480 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.001464 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e5215ace-4db9-42de-ac5d-a0a68738cbbb-agent-certs\") pod \"konnectivity-agent-7nrcw\" (UID: \"e5215ace-4db9-42de-ac5d-a0a68738cbbb\") " pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:17.005782 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.005764 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7829\" (UniqueName: \"kubernetes.io/projected/02228a0a-bfe5-4615-b4f2-45dfd3cd66e6-kube-api-access-j7829\") pod \"multus-k7ngj\" (UID: \"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6\") " pod="openshift-multus/multus-k7ngj" Jun 09 14:12:17.098977 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.098902 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" Jun 09 14:12:17.105281 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.105260 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" Jun 09 14:12:17.109355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.109330 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6f3949_59cd_4c78_8709_b027e61dea18.slice/crio-12d1907633d4492965d394dfff8a43f07cce0588a652bb87f9d8663279fba237 WatchSource:0}: Error finding container 12d1907633d4492965d394dfff8a43f07cce0588a652bb87f9d8663279fba237: Status 404 returned error can't find the container with id 12d1907633d4492965d394dfff8a43f07cce0588a652bb87f9d8663279fba237 Jun 09 14:12:17.113253 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.113231 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f56ab11_bfb5_416f_99e7_0b29e8260701.slice/crio-a0f75b400f0481dda2d20e0deaf6ec7c96621b5e47a2b5ab4bd75e6f8bc7dca8 WatchSource:0}: Error finding container a0f75b400f0481dda2d20e0deaf6ec7c96621b5e47a2b5ab4bd75e6f8bc7dca8: Status 404 returned error can't find the container with id a0f75b400f0481dda2d20e0deaf6ec7c96621b5e47a2b5ab4bd75e6f8bc7dca8 Jun 09 14:12:17.121679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.121658 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4zh7k" Jun 09 14:12:17.130180 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.130158 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d454a83_cb18_4aa5_82b7_73fa3cfed11c.slice/crio-0bd1f4c1f23b47bd9e7734aa7ccf45ab74e723a6b24eb4efd5c17105801b1ef1 WatchSource:0}: Error finding container 0bd1f4c1f23b47bd9e7734aa7ccf45ab74e723a6b24eb4efd5c17105801b1ef1: Status 404 returned error can't find the container with id 0bd1f4c1f23b47bd9e7734aa7ccf45ab74e723a6b24eb4efd5c17105801b1ef1 Jun 09 14:12:17.135261 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.135242 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" Jun 09 14:12:17.145359 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.145333 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45354e28_54ab_460d_981a_86b17ed5cce9.slice/crio-44f32eba04e2e733265e019cf147b14573ad2a567cd8d77cf86699cba393250f WatchSource:0}: Error finding container 44f32eba04e2e733265e019cf147b14573ad2a567cd8d77cf86699cba393250f: Status 404 returned error can't find the container with id 44f32eba04e2e733265e019cf147b14573ad2a567cd8d77cf86699cba393250f Jun 09 14:12:17.147352 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.147276 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rk89k" Jun 09 14:12:17.155039 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.155014 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da21813_284f_4169_b06c_dd9b636cdc8f.slice/crio-ee41228fc044801362b5d1747b42d671988eb1bd317f286bb5f3f54e6324f6e9 WatchSource:0}: Error finding container ee41228fc044801362b5d1747b42d671988eb1bd317f286bb5f3f54e6324f6e9: Status 404 returned error can't find the container with id ee41228fc044801362b5d1747b42d671988eb1bd317f286bb5f3f54e6324f6e9 Jun 09 14:12:17.171511 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.171482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vvsn4" Jun 09 14:12:17.179313 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.179291 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18c19c3_e11d_4191_81a4_73707986bee7.slice/crio-d57581d4cdba7d3d9aa25466cb5945d2999afabaefa7c564b159ab6079296c48 WatchSource:0}: Error finding container d57581d4cdba7d3d9aa25466cb5945d2999afabaefa7c564b159ab6079296c48: Status 404 returned error can't find the container with id d57581d4cdba7d3d9aa25466cb5945d2999afabaefa7c564b159ab6079296c48 Jun 09 14:12:17.191618 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.191596 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:17.199505 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.199482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k7ngj" Jun 09 14:12:17.199999 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.199976 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8393f4ed_5c8d_4685_8bc0_87d143940155.slice/crio-912715aeec597c1576e7afb6f0599044c80510426a4d25c311d9d563d7f13893 WatchSource:0}: Error finding container 912715aeec597c1576e7afb6f0599044c80510426a4d25c311d9d563d7f13893: Status 404 returned error can't find the container with id 912715aeec597c1576e7afb6f0599044c80510426a4d25c311d9d563d7f13893 Jun 09 14:12:17.208299 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.208273 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02228a0a_bfe5_4615_b4f2_45dfd3cd66e6.slice/crio-41e0fa526c80e912c09373cb5164bff1f237d181ad568742a7b6413bd5cbb13a WatchSource:0}: Error finding container 41e0fa526c80e912c09373cb5164bff1f237d181ad568742a7b6413bd5cbb13a: Status 404 returned error can't find the container with id 41e0fa526c80e912c09373cb5164bff1f237d181ad568742a7b6413bd5cbb13a Jun 09 14:12:17.219643 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.219620 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:17.227860 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:17.227834 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5215ace_4db9_42de_ac5d_a0a68738cbbb.slice/crio-53af67fc49b9acea90bc7c5fa3ece8eabd4496de5d123fb01aa3c82c7763ce9d WatchSource:0}: Error finding container 53af67fc49b9acea90bc7c5fa3ece8eabd4496de5d123fb01aa3c82c7763ce9d: Status 404 returned error can't find the container with id 53af67fc49b9acea90bc7c5fa3ece8eabd4496de5d123fb01aa3c82c7763ce9d Jun 09 14:12:17.402202 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.402118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:17.402202 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.402162 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:17.402389 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.402269 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:17.402389 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.402275 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:17.402389 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.402339 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:12:18.402319682 +0000 UTC m=+3.049066372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:17.402389 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.402361 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:12:18.40235106 +0000 UTC m=+3.049097737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:17.502958 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.502921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:17.503129 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.503098 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:17.503129 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.503119 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:17.503218 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.503132 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:17.503218 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.503190 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:18.503172075 +0000 UTC m=+3.149918765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:17.628814 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.628780 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:17.829957 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.829867 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 14:07:16 +0000 UTC" deadline="2028-03-06 18:40:34.099519772 +0000 UTC" Jun 09 14:12:17.829957 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.829931 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15268h28m16.269593061s" Jun 09 14:12:17.884509 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.872644 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:17.884509 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.872650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:17.884509 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.873046 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:17.884509 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:17.873148 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:17.891804 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.891700 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" event={"ID":"a72335425bac515a27766cdcfbe29358","Type":"ContainerStarted","Data":"dd34b5d2fa6883ab2846983967f62c7d6225da0edbf78da246845f9da216c55a"} Jun 09 14:12:17.902016 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.901974 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" event={"ID":"3ad19473c8d2169042690ba1f43febb8","Type":"ContainerStarted","Data":"fbd1df25cab61201f54d78276228b7946af8fa83b1c793dbda0ee1c9b7319020"} Jun 09 14:12:17.906700 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.906664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7nrcw" event={"ID":"e5215ace-4db9-42de-ac5d-a0a68738cbbb","Type":"ContainerStarted","Data":"53af67fc49b9acea90bc7c5fa3ece8eabd4496de5d123fb01aa3c82c7763ce9d"} Jun 09 14:12:17.914507 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.914467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" event={"ID":"45354e28-54ab-460d-981a-86b17ed5cce9","Type":"ContainerStarted","Data":"44f32eba04e2e733265e019cf147b14573ad2a567cd8d77cf86699cba393250f"} Jun 09 14:12:17.934544 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.934500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4zh7k" event={"ID":"8d454a83-cb18-4aa5-82b7-73fa3cfed11c","Type":"ContainerStarted","Data":"0bd1f4c1f23b47bd9e7734aa7ccf45ab74e723a6b24eb4efd5c17105801b1ef1"} Jun 09 14:12:17.958256 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.958170 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" event={"ID":"9f56ab11-bfb5-416f-99e7-0b29e8260701","Type":"ContainerStarted","Data":"a0f75b400f0481dda2d20e0deaf6ec7c96621b5e47a2b5ab4bd75e6f8bc7dca8"} Jun 09 14:12:17.961541 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.961450 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k7ngj" event={"ID":"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6","Type":"ContainerStarted","Data":"41e0fa526c80e912c09373cb5164bff1f237d181ad568742a7b6413bd5cbb13a"} Jun 09 14:12:17.967784 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.967534 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"912715aeec597c1576e7afb6f0599044c80510426a4d25c311d9d563d7f13893"} Jun 09 14:12:17.977226 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.975095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vvsn4" event={"ID":"a18c19c3-e11d-4191-81a4-73707986bee7","Type":"ContainerStarted","Data":"d57581d4cdba7d3d9aa25466cb5945d2999afabaefa7c564b159ab6079296c48"} Jun 09 14:12:17.980253 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.980170 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rk89k" event={"ID":"8da21813-284f-4169-b06c-dd9b636cdc8f","Type":"ContainerStarted","Data":"ee41228fc044801362b5d1747b42d671988eb1bd317f286bb5f3f54e6324f6e9"} Jun 09 14:12:17.995698 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:17.995655 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerStarted","Data":"12d1907633d4492965d394dfff8a43f07cce0588a652bb87f9d8663279fba237"} Jun 09 14:12:18.015925 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.015890 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:18.409968 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.409932 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:18.410158 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.409991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:18.410158 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.410149 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:18.410334 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.410232 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:12:20.410211772 +0000 UTC m=+5.056958448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:18.410692 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.410663 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:18.410774 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.410724 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:12:20.410708539 +0000 UTC m=+5.057455214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:18.511022 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.510983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:18.511216 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.511159 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:18.511216 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.511180 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:18.511216 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.511201 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:18.511441 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.511259 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:20.511240891 +0000 UTC m=+5.157987565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:18.830799 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.830750 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 14:07:16 +0000 UTC" deadline="2028-02-20 05:31:16.1009055 +0000 UTC" Jun 09 14:12:18.830799 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.830788 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14895h18m57.270122208s" Jun 09 14:12:18.871230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:18.871195 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:18.871513 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:18.871467 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:19.630120 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:19.630088 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 14:12:19.873512 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:19.873481 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:19.874055 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:19.873628 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:19.874195 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:19.874171 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:19.874305 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:19.874279 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:20.428417 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:20.428352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:20.428615 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:20.428463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:20.428615 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.428588 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:20.428739 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.428651 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:12:24.428630751 +0000 UTC m=+9.075377421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:20.429076 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.429058 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:20.429152 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.429110 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:12:24.429095742 +0000 UTC m=+9.075842415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:20.529695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:20.529659 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:20.529926 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.529906 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:20.530019 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.529930 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:20.530019 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.529945 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:20.530019 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.530003 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:24.529983231 +0000 UTC m=+9.176729907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:20.872010 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:20.871510 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:20.872010 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:20.871651 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:21.874197 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:21.874161 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:21.874664 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:21.874305 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:21.874747 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:21.874718 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:21.874838 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:21.874811 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:22.871800 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:22.871759 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:22.871983 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:22.871892 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:23.871510 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:23.871468 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:23.871510 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:23.871498 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:23.871974 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:23.871671 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:23.871974 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:23.871811 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:24.464421 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:24.464372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:24.464620 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:24.464437 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:24.464620 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.464556 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:24.464620 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.464594 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:24.464772 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.464642 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:12:32.46462045 +0000 UTC m=+17.111367139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:24.464772 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.464662 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:12:32.464652083 +0000 UTC m=+17.111398753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:24.565427 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:24.565326 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:24.565584 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.565544 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:24.565584 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.565572 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:24.565663 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.565587 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:24.565663 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.565650 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:32.565630944 +0000 UTC m=+17.212377621 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:24.871934 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:24.871897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:24.872367 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:24.872034 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:25.872183 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:25.872155 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:25.872640 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:25.872257 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:25.872640 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:25.872340 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:25.872640 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:25.872477 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:26.871367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:26.871326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:26.871577 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:26.871488 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:27.872537 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:27.872504 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:27.873074 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:27.872549 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:27.873074 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:27.872641 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:27.873074 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:27.872743 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:28.871839 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:28.871801 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:28.871998 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:28.871910 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:29.872531 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:29.871863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:29.872531 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:29.872017 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:29.872531 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:29.871863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:29.872531 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:29.872485 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:30.871514 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:30.871476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:30.871706 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:30.871613 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:31.871521 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:31.871482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:31.871975 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:31.871618 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:31.871975 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:31.871482 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:31.872090 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:31.872073 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:32.524877 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:32.524832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:32.525094 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:32.524911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:32.525094 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.525028 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:32.525094 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.525073 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:32.525256 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.525116 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:12:48.525092118 +0000 UTC m=+33.171838789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:32.525256 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.525150 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:12:48.525135613 +0000 UTC m=+33.171882288 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:32.625990 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:32.625935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:32.626180 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.626121 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:32.626180 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.626143 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:32.626180 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.626153 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:32.626305 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.626214 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:48.626195706 +0000 UTC m=+33.272942395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:32.871040 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:32.870959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:32.871190 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:32.871095 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:33.871616 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:33.871580 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:33.872044 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:33.871702 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:33.872044 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:33.872033 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:33.872146 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:33.872114 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:34.871913 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:34.871874 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:34.872345 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:34.872008 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:35.871694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:35.871665 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:35.871861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:35.871716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:35.871861 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:35.871804 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:35.871962 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:35.871928 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:36.871733 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:36.871694 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:36.871911 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:36.871819 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:37.872172 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:37.871974 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:37.872590 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:37.871994 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:37.872590 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:37.872296 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:37.872590 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:37.872396 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:38.059601 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.059518 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" event={"ID":"45354e28-54ab-460d-981a-86b17ed5cce9","Type":"ContainerStarted","Data":"83084e7e461269b8826d4ee3995512141470974e9e1956c592297b0e2e4eda58"} Jun 09 14:12:38.061515 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.061459 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k7ngj" event={"ID":"02228a0a-bfe5-4615-b4f2-45dfd3cd66e6","Type":"ContainerStarted","Data":"e5c066f0cd4fa72a8da750fc43e7a59fa0081bb6fa3edf217b9f8c739e1bf03d"} Jun 09 14:12:38.064322 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.064295 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:12:38.064876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.064850 2568 generic.go:356] "Generic (PLEG): container finished" podID="8393f4ed-5c8d-4685-8bc0-87d143940155" containerID="4682a67dcefb3f8f3ae04875cf593486684333f045e74fefd759b459daa929f0" exitCode=1 Jun 09 14:12:38.064954 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.064922 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"a31438e7ab49ddc2201320aee624e0a72c499bc87731b091e26dad42a332a536"} Jun 09 14:12:38.065014 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.065001 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerDied","Data":"4682a67dcefb3f8f3ae04875cf593486684333f045e74fefd759b459daa929f0"} Jun 09 14:12:38.065062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.065042 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"bbf32f7128a9efaaca44d4afe22b4b8d1020039e840f9f50a5bcd9a90d6d62e4"} Jun 09 14:12:38.066530 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.066503 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" event={"ID":"a72335425bac515a27766cdcfbe29358","Type":"ContainerStarted","Data":"f57a2610149b0825e798a000e2db51ffe94737c60a76327b60151ed107bde8bd"} Jun 09 14:12:38.076920 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.076873 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7dc9x" podStartSLOduration=1.557127349 podStartE2EDuration="22.076858611s" totalImagesPullingTime="20.519731262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:38.075674253 +0000 UTC m=+22.722420945" watchObservedRunningTime="2026-06-09 14:12:38.076858611 +0000 UTC m=+22.723605304" Jun 09 14:12:38.091427 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.091363 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k7ngj" podStartSLOduration=1.541288401 podStartE2EDuration="22.091346909s" totalImagesPullingTime="20.550058508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:38.090777012 +0000 UTC m=+22.737523705" watchObservedRunningTime="2026-06-09 14:12:38.091346909 +0000 UTC m=+22.738093602" Jun 09 14:12:38.104114 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.104051 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-214.ec2.internal" podStartSLOduration=22.104026812 podStartE2EDuration="22.104026812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:38.103374295 +0000 UTC m=+22.750120988" watchObservedRunningTime="2026-06-09 14:12:38.104026812 +0000 UTC m=+22.750773589" Jun 09 14:12:38.871802 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:38.871609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:38.871983 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:38.871878 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:39.070665 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.070620 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rk89k" event={"ID":"8da21813-284f-4169-b06c-dd9b636cdc8f","Type":"ContainerStarted","Data":"241340af67c43a412916a0b6e6d7c750ba74681b7ccc5189e8b7babe503c3635"} Jun 09 14:12:39.072818 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.072786 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f6f3949-59cd-4c78-8709-b027e61dea18" containerID="c66cd28725d594dcecc963e93f7d71619aac53c63d938dffbb20bc716c45f810" exitCode=0 Jun 09 14:12:39.072970 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.072860 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerDied","Data":"c66cd28725d594dcecc963e93f7d71619aac53c63d938dffbb20bc716c45f810"} Jun 09 14:12:39.074481 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.074451 2568 generic.go:356] "Generic (PLEG): container finished" podID="3ad19473c8d2169042690ba1f43febb8" containerID="3073709431fb14bd2fa52bc3037b7f47ad89cf2d4d747210e5bfc307aaeda177" exitCode=0 Jun 09 14:12:39.074594 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.074533 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" event={"ID":"3ad19473c8d2169042690ba1f43febb8","Type":"ContainerDied","Data":"3073709431fb14bd2fa52bc3037b7f47ad89cf2d4d747210e5bfc307aaeda177"} Jun 09 14:12:39.076338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.076286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7nrcw" event={"ID":"e5215ace-4db9-42de-ac5d-a0a68738cbbb","Type":"ContainerStarted","Data":"d6e6a6ff3596737b69b33624e63010c347d682525539a1dbb001318be7d95d77"} Jun 09 14:12:39.078248 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.078214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4zh7k" event={"ID":"8d454a83-cb18-4aa5-82b7-73fa3cfed11c","Type":"ContainerStarted","Data":"62fb519ad2357c2a2f61ee81ee0a692703b2eb177fb3bfeca4fe1fe8ffd076a0"} Jun 09 14:12:39.079834 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.079807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" event={"ID":"9f56ab11-bfb5-416f-99e7-0b29e8260701","Type":"ContainerStarted","Data":"fbfc89eb237955d85ed2942068bcc4157ca402b934f7dea2791b312ec77021e3"} Jun 09 14:12:39.082723 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.082704 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:12:39.083115 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.083092 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"a918bd0b00682a65deeec33fea5dc7b04da4a6fabe88e53ed70d4a84b0157d87"} Jun 09 14:12:39.083216 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.083159 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"78106d68f89b9e74303cf2853d5c9d9249639f105d6dafa39189b18435754a6c"} Jun 09 14:12:39.083216 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.083190 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"91cc3750f42c7ef986f9b61d287e8b689b31b52aa9bb32355171f2437b816212"} Jun 09 14:12:39.084880 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.084857 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vvsn4" event={"ID":"a18c19c3-e11d-4191-81a4-73707986bee7","Type":"ContainerStarted","Data":"6658b0b62500e30f97e5a3e1498a08903ef30cf9c81af8216f20dcc71571b29d"} Jun 09 14:12:39.084982 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.084926 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rk89k" podStartSLOduration=2.574974763 podStartE2EDuration="23.084912586s" totalImagesPullingTime="20.509937823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:39.084789637 +0000 UTC m=+23.731536331" watchObservedRunningTime="2026-06-09 14:12:39.084912586 +0000 UTC m=+23.731659279" Jun 09 14:12:39.098580 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.098535 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4zh7k" podStartSLOduration=3.563958381 podStartE2EDuration="24.098519509s" totalImagesPullingTime="20.534561128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:39.097790874 +0000 UTC m=+23.744537566" watchObservedRunningTime="2026-06-09 14:12:39.098519509 +0000 UTC m=+23.745266202" Jun 09 14:12:39.146997 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.146898 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7nrcw" podStartSLOduration=2.70978274 podStartE2EDuration="23.146882441s" totalImagesPullingTime="20.437099701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:39.14657227 +0000 UTC m=+23.793318965" watchObservedRunningTime="2026-06-09 14:12:39.146882441 +0000 UTC m=+23.793629133" Jun 09 14:12:39.163313 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.163258 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vvsn4" podStartSLOduration=11.287048628 podStartE2EDuration="23.163233659s" totalImagesPullingTime="11.876185031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:39.162833813 +0000 UTC m=+23.809580507" watchObservedRunningTime="2026-06-09 14:12:39.163233659 +0000 UTC m=+23.809980354" Jun 09 14:12:39.208468 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.208436 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:39.290208 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.290172 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:39.870522 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.870444 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 14:12:39.870702 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.870513 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T14:12:39.870465185Z","UUID":"de68dd85-6a54-4bab-979b-c658650ae052","Handler":null,"Name":"","Endpoint":""} Jun 09 14:12:39.871993 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.871962 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 14:12:39.872128 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.872000 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 14:12:39.874593 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.874574 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:39.874717 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:39.874577 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:39.874717 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:39.874667 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:39.874795 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:39.874777 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:40.089381 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:40.089344 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" event={"ID":"9f56ab11-bfb5-416f-99e7-0b29e8260701","Type":"ContainerStarted","Data":"714a38f88efb0e342c3fed894fa8092bfcd4457e1a05f1958cfc853570b4b2e2"} Jun 09 14:12:40.091497 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:40.091446 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" event={"ID":"3ad19473c8d2169042690ba1f43febb8","Type":"ContainerStarted","Data":"ffd96bde7333d729a104977c093a2e76486225e7715a7063f7074f29edda2fbd"} Jun 09 14:12:40.106362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:40.106309 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-214.ec2.internal" podStartSLOduration=24.106294963 podStartE2EDuration="24.106294963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:40.106082605 +0000 UTC m=+24.752829303" watchObservedRunningTime="2026-06-09 14:12:40.106294963 +0000 UTC m=+24.753041656" Jun 09 14:12:40.871034 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:40.871002 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:40.871232 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:40.871134 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:41.097916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:41.097890 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:12:41.098316 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:41.098288 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"3d4e9c399a83fcdea7a5d1625108ec36ab490184292480c107a61859195d9773"} Jun 09 14:12:41.871150 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:41.870964 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:41.871330 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:41.870953 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:41.871330 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:41.871231 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:41.871330 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:41.871310 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:42.034609 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.034569 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:42.035921 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.035891 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:42.102196 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.102157 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" event={"ID":"9f56ab11-bfb5-416f-99e7-0b29e8260701","Type":"ContainerStarted","Data":"9ffcddd9bb71104b124d1be4d2bedab4d7da86e5301345c454a81492b76e5cc2"} Jun 09 14:12:42.102680 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.102551 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:42.102962 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.102944 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7nrcw" Jun 09 14:12:42.119167 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.119104 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rkdmj" podStartSLOduration=4.262839599 podStartE2EDuration="27.119083545s" totalImagesPullingTime="22.856243946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:42.118968082 +0000 UTC m=+26.765714775" watchObservedRunningTime="2026-06-09 14:12:42.119083545 +0000 UTC m=+26.765830242" Jun 09 14:12:42.207575 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.207371 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:42.412826 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.412790 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:42.871911 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:42.871836 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:42.872072 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:42.871945 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:43.871746 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:43.871720 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:43.872465 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:43.871867 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:43.872465 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:43.871914 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:43.872465 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:43.872074 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:44.110205 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.110182 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:12:44.110604 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.110579 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"cf5a2b38a39a5bb8391e87fb3e017c4f6b04accfdbff9f8d8817dc33d95653af"} Jun 09 14:12:44.110872 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.110837 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:44.111031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.111011 2568 scope.go:117] "RemoveContainer" containerID="4682a67dcefb3f8f3ae04875cf593486684333f045e74fefd759b459daa929f0" Jun 09 14:12:44.112528 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.112505 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f6f3949-59cd-4c78-8709-b027e61dea18" containerID="b9418e556fb3e8e9fbd9f2fa0c8f4099fdbf46132c6a50a2160aa978c51c6b55" exitCode=0 Jun 09 14:12:44.112636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.112584 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerDied","Data":"b9418e556fb3e8e9fbd9f2fa0c8f4099fdbf46132c6a50a2160aa978c51c6b55"} Jun 09 14:12:44.130590 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.130570 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:44.228511 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.228461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:44.280128 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.280091 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:44.871628 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:44.871594 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:44.871868 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:44.871711 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:45.118774 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.118749 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:12:45.119133 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.119109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" event={"ID":"8393f4ed-5c8d-4685-8bc0-87d143940155","Type":"ContainerStarted","Data":"df01229e7d8f75bc9c9b78aa410580a125e100e9b8c3beed0c223436cedd99b2"} Jun 09 14:12:45.119418 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.119382 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:45.119522 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.119508 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:45.135437 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.135356 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:12:45.148227 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.148177 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" podStartSLOduration=8.602902952 podStartE2EDuration="29.148162122s" totalImagesPullingTime="20.54525917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:45.146390821 +0000 UTC m=+29.793137517" watchObservedRunningTime="2026-06-09 14:12:45.148162122 +0000 UTC m=+29.794908814" Jun 09 14:12:45.485727 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.485635 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hsxpw"] Jun 09 14:12:45.485915 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.485785 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:45.485982 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:45.485906 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:45.486448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.486424 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lsq9c"] Jun 09 14:12:45.489196 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.486811 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:45.489196 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:45.486960 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:45.496824 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.496802 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bcb8m"] Jun 09 14:12:45.496935 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:45.496920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:45.497015 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:45.496997 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:46.123146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:46.123111 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f6f3949-59cd-4c78-8709-b027e61dea18" containerID="ee53b93fa78f5bf9a9ba9afb2085cf7622da88c0f886308a01e11f23eac1c449" exitCode=0 Jun 09 14:12:46.123583 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:46.123201 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerDied","Data":"ee53b93fa78f5bf9a9ba9afb2085cf7622da88c0f886308a01e11f23eac1c449"} Jun 09 14:12:46.871096 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:46.871002 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:46.871249 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:46.871138 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:46.871315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:46.871302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:46.871481 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:46.871455 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:47.871300 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:47.871258 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:47.871934 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:47.871366 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:48.130173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.130084 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f6f3949-59cd-4c78-8709-b027e61dea18" containerID="594e7d9257491e9f8053f69dc46e63ab8cbd9f95026330fbb83f1cd36d26b8c9" exitCode=0 Jun 09 14:12:48.130173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.130134 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerDied","Data":"594e7d9257491e9f8053f69dc46e63ab8cbd9f95026330fbb83f1cd36d26b8c9"} Jun 09 14:12:48.547865 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.547830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:48.548060 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.547899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:48.548060 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.547995 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:48.548060 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.548001 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:48.548228 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.548069 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret podName:4a611bc1-9997-42fa-88ae-2feb6105e7bd nodeName:}" failed. No retries permitted until 2026-06-09 14:13:20.54804883 +0000 UTC m=+65.194795500 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret") pod "global-pull-secret-syncer-bcb8m" (UID: "4a611bc1-9997-42fa-88ae-2feb6105e7bd") : object "kube-system"/"original-pull-secret" not registered Jun 09 14:12:48.548228 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.548089 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:13:20.548079285 +0000 UTC m=+65.194825957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 14:12:48.648871 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.648836 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:48.649045 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.648975 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 14:12:48.649045 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.648990 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 14:12:48.649045 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.649001 2568 projected.go:196] Error preparing data for projected volume kube-api-access-r8mq2 for pod openshift-network-diagnostics/network-check-target-lsq9c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:48.649166 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.649050 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2 podName:7f73adfd-5798-4afb-bf74-2f5ab7462587 nodeName:}" failed. No retries permitted until 2026-06-09 14:13:20.649036134 +0000 UTC m=+65.295782805 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8mq2" (UniqueName: "kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2") pod "network-check-target-lsq9c" (UID: "7f73adfd-5798-4afb-bf74-2f5ab7462587") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 14:12:48.871098 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.871062 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:48.871289 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:48.871059 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:48.871289 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.871209 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:12:48.871289 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:48.871231 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bcb8m" podUID="4a611bc1-9997-42fa-88ae-2feb6105e7bd" Jun 09 14:12:49.450232 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:49.450203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:49.502032 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:49.501997 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:49.871658 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:49.871628 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:49.872034 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:49.871752 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lsq9c" podUID="7f73adfd-5798-4afb-bf74-2f5ab7462587" Jun 09 14:12:50.627202 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.627121 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-214.ec2.internal" event="NodeReady" Jun 09 14:12:50.627367 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.627294 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 14:12:50.670826 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.670791 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-54mcq"] Jun 09 14:12:50.698661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.698628 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hvcpz"] Jun 09 14:12:50.698818 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.698761 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.701235 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.701207 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 14:12:50.701235 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.701231 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-55pg5\"" Jun 09 14:12:50.701456 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.701307 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 14:12:50.724084 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.723892 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-54mcq"] Jun 09 14:12:50.724189 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.724020 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:50.724312 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.724297 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hvcpz"] Jun 09 14:12:50.726689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.726514 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 14:12:50.726689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.726557 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dqnb4\"" Jun 09 14:12:50.726689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.726605 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 14:12:50.726689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.726623 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 14:12:50.765022 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.764986 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a06ae45d-d42d-4ed3-9770-95289acac343-config-volume\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.765159 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.765027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/a06ae45d-d42d-4ed3-9770-95289acac343-kube-api-access-52wgk\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.765159 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.765061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06ae45d-d42d-4ed3-9770-95289acac343-tmp-dir\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.765272 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.765164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.782452 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.782416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:50.865653 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.865614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a06ae45d-d42d-4ed3-9770-95289acac343-config-volume\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.865822 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.865659 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6221db5-4346-494c-85c3-9cf89b8cba98-cert\") pod \"ingress-canary-hvcpz\" (UID: \"d6221db5-4346-494c-85c3-9cf89b8cba98\") " pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:50.865822 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.865690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/a06ae45d-d42d-4ed3-9770-95289acac343-kube-api-access-52wgk\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.865822 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.865720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06ae45d-d42d-4ed3-9770-95289acac343-tmp-dir\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.865994 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.865892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpd5j\" (UniqueName: \"kubernetes.io/projected/d6221db5-4346-494c-85c3-9cf89b8cba98-kube-api-access-tpd5j\") pod \"ingress-canary-hvcpz\" (UID: \"d6221db5-4346-494c-85c3-9cf89b8cba98\") " pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:50.865994 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.865945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.866095 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.866050 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a06ae45d-d42d-4ed3-9770-95289acac343-tmp-dir\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.866095 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:50.866068 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:12:50.866188 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:50.866134 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:51.366115842 +0000 UTC m=+36.012862518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:12:50.866284 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.866266 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a06ae45d-d42d-4ed3-9770-95289acac343-config-volume\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.871587 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.871563 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:12:50.871701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.871601 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:12:50.873932 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.873904 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 14:12:50.874036 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.873983 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 14:12:50.874107 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.874092 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xvhx5\"" Jun 09 14:12:50.876466 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.876446 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/a06ae45d-d42d-4ed3-9770-95289acac343-kube-api-access-52wgk\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:50.881933 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.881862 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:50.966388 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.966358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpd5j\" (UniqueName: \"kubernetes.io/projected/d6221db5-4346-494c-85c3-9cf89b8cba98-kube-api-access-tpd5j\") pod \"ingress-canary-hvcpz\" (UID: \"d6221db5-4346-494c-85c3-9cf89b8cba98\") " pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:50.966568 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.966422 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6221db5-4346-494c-85c3-9cf89b8cba98-cert\") pod \"ingress-canary-hvcpz\" (UID: \"d6221db5-4346-494c-85c3-9cf89b8cba98\") " pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:50.969584 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.969556 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6221db5-4346-494c-85c3-9cf89b8cba98-cert\") pod \"ingress-canary-hvcpz\" (UID: \"d6221db5-4346-494c-85c3-9cf89b8cba98\") " pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:50.974568 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:50.974535 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpd5j\" (UniqueName: \"kubernetes.io/projected/d6221db5-4346-494c-85c3-9cf89b8cba98-kube-api-access-tpd5j\") pod \"ingress-canary-hvcpz\" (UID: \"d6221db5-4346-494c-85c3-9cf89b8cba98\") " pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:51.035505 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.035474 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hvcpz" Jun 09 14:12:51.257559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.257532 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hvcpz"] Jun 09 14:12:51.262104 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:12:51.262077 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6221db5_4346_494c_85c3_9cf89b8cba98.slice/crio-8903846f9164f53f5aac8308c952802bd13be6a74ffabbbfc17d4bd1c55f15ec WatchSource:0}: Error finding container 8903846f9164f53f5aac8308c952802bd13be6a74ffabbbfc17d4bd1c55f15ec: Status 404 returned error can't find the container with id 8903846f9164f53f5aac8308c952802bd13be6a74ffabbbfc17d4bd1c55f15ec Jun 09 14:12:51.369842 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.369805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:51.370033 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:51.369977 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:12:51.370086 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:51.370052 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:52.370034871 +0000 UTC m=+37.016781542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:12:51.871922 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.871889 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:12:51.874660 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.874623 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 14:12:51.874804 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.874690 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 14:12:51.874804 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:51.874707 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hlvfh\"" Jun 09 14:12:52.143048 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:52.142967 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hvcpz" event={"ID":"d6221db5-4346-494c-85c3-9cf89b8cba98","Type":"ContainerStarted","Data":"8903846f9164f53f5aac8308c952802bd13be6a74ffabbbfc17d4bd1c55f15ec"} Jun 09 14:12:52.377569 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:52.377534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:52.377727 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:52.377680 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:12:52.377768 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:52.377744 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:54.377726715 +0000 UTC m=+39.024473402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:12:54.394141 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:54.394092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:54.394652 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:54.394304 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:12:54.394652 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:54.394382 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:12:58.394360491 +0000 UTC m=+43.041107182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:12:55.154445 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:55.154182 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f6f3949-59cd-4c78-8709-b027e61dea18" containerID="2badb7dfd3536855c7e505d7c0bee280daf6650871b5f835b96eb2959e2709ed" exitCode=0 Jun 09 14:12:55.154617 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:55.154261 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerDied","Data":"2badb7dfd3536855c7e505d7c0bee280daf6650871b5f835b96eb2959e2709ed"} Jun 09 14:12:55.155980 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:55.155954 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hvcpz" event={"ID":"d6221db5-4346-494c-85c3-9cf89b8cba98","Type":"ContainerStarted","Data":"ed69b7c939b932bf423546a212035df47c2024ce65bf3efb04ffe10b6cbefa2e"} Jun 09 14:12:55.188037 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:55.187814 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hvcpz" podStartSLOduration=1.754726404 podStartE2EDuration="5.187795652s" totalImagesPullingTime="3.433069248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:55.187095797 +0000 UTC m=+39.833842489" watchObservedRunningTime="2026-06-09 14:12:55.187795652 +0000 UTC m=+39.834542346" Jun 09 14:12:55.896213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:55.896177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:55.997365 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:55.997329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:56.161347 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:56.161267 2568 generic.go:356] "Generic (PLEG): container finished" podID="2f6f3949-59cd-4c78-8709-b027e61dea18" containerID="d90474bd7e2ca43bac486a926330d77be851e7d46d7586b47f4d85a6ece6c8d2" exitCode=0 Jun 09 14:12:56.161499 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:56.161346 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerDied","Data":"d90474bd7e2ca43bac486a926330d77be851e7d46d7586b47f4d85a6ece6c8d2"} Jun 09 14:12:56.254705 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:56.254676 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:56.355077 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:56.355044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:57.167113 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:57.167077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" event={"ID":"2f6f3949-59cd-4c78-8709-b027e61dea18","Type":"ContainerStarted","Data":"d01a9fd581bdbfa20e606e145ae51146400c0d968fbfb8dcbdb83232f273cb1c"} Jun 09 14:12:57.189711 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:57.189663 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mkqk7" podStartSLOduration=8.131070964 podStartE2EDuration="42.189648165s" totalImagesPullingTime="34.058577201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:12:57.188030623 +0000 UTC m=+41.834777317" watchObservedRunningTime="2026-06-09 14:12:57.189648165 +0000 UTC m=+41.836394848" Jun 09 14:12:57.308867 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:57.308833 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:57.361778 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:57.361741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:12:58.426595 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:12:58.426555 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:12:58.427037 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:58.426684 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:12:58.427037 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:12:58.426736 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:13:06.426722451 +0000 UTC m=+51.073469122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:13:05.358200 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:05.358166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:13:05.407959 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:05.407929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:13:06.474110 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:06.474074 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:13:06.474539 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:06.474205 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:13:06.474539 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:06.474259 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:13:22.474244602 +0000 UTC m=+67.120991273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:13:17.136929 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:17.136900 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2wmnf" Jun 09 14:13:17.256657 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:17.256628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:13:17.356982 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:17.356953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:13:17.406027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:17.405935 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:13:20.570155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.570107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:13:20.570155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.570162 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:13:20.572686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.572656 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 14:13:20.572809 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.572715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 14:13:20.580614 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:20.580596 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 14:13:20.580669 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:20.580659 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:14:24.580642149 +0000 UTC m=+129.227388823 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : secret "metrics-daemon-secret" not found Jun 09 14:13:20.584154 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.584125 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4a611bc1-9997-42fa-88ae-2feb6105e7bd-original-pull-secret\") pod \"global-pull-secret-syncer-bcb8m\" (UID: \"4a611bc1-9997-42fa-88ae-2feb6105e7bd\") " pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:13:20.592175 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.592156 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bcb8m" Jun 09 14:13:20.670582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.670555 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:13:20.684886 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.684856 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 14:13:20.693225 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.693201 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 14:13:20.704615 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.704585 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8mq2\" (UniqueName: \"kubernetes.io/projected/7f73adfd-5798-4afb-bf74-2f5ab7462587-kube-api-access-r8mq2\") pod \"network-check-target-lsq9c\" (UID: \"7f73adfd-5798-4afb-bf74-2f5ab7462587\") " pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:13:20.713948 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.713923 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bcb8m"] Jun 09 14:13:20.717229 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:13:20.717207 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a611bc1_9997_42fa_88ae_2feb6105e7bd.slice/crio-538819a83d5d77b0afaa13537e0e06decf659dffbc432bf519dd7315df1ac952 WatchSource:0}: Error finding container 538819a83d5d77b0afaa13537e0e06decf659dffbc432bf519dd7315df1ac952: Status 404 returned error can't find the container with id 538819a83d5d77b0afaa13537e0e06decf659dffbc432bf519dd7315df1ac952 Jun 09 14:13:20.987172 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.987149 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hlvfh\"" Jun 09 14:13:20.995087 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:20.995068 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:13:21.128281 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:21.128246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lsq9c"] Jun 09 14:13:21.132341 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:13:21.132314 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f73adfd_5798_4afb_bf74_2f5ab7462587.slice/crio-925b45b6c2d02668eeb27e6176110a957770c06c5ffe908931999bb43bddaaa9 WatchSource:0}: Error finding container 925b45b6c2d02668eeb27e6176110a957770c06c5ffe908931999bb43bddaaa9: Status 404 returned error can't find the container with id 925b45b6c2d02668eeb27e6176110a957770c06c5ffe908931999bb43bddaaa9 Jun 09 14:13:21.223655 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:21.223622 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lsq9c" event={"ID":"7f73adfd-5798-4afb-bf74-2f5ab7462587","Type":"ContainerStarted","Data":"925b45b6c2d02668eeb27e6176110a957770c06c5ffe908931999bb43bddaaa9"} Jun 09 14:13:21.224810 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:21.224783 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bcb8m" event={"ID":"4a611bc1-9997-42fa-88ae-2feb6105e7bd","Type":"ContainerStarted","Data":"538819a83d5d77b0afaa13537e0e06decf659dffbc432bf519dd7315df1ac952"} Jun 09 14:13:22.485053 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:22.485020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:13:22.485538 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:22.485148 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:13:22.485538 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:22.485215 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:13:54.485199714 +0000 UTC m=+99.131946386 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:13:26.239850 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:26.239816 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bcb8m" event={"ID":"4a611bc1-9997-42fa-88ae-2feb6105e7bd","Type":"ContainerStarted","Data":"0098e419ef7e082b829183b6755062a48bf6699dca7e30433e1ceedf0f102c22"} Jun 09 14:13:26.241263 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:26.241240 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lsq9c" event={"ID":"7f73adfd-5798-4afb-bf74-2f5ab7462587","Type":"ContainerStarted","Data":"1b99b77563227847a75976c2367b81bf192416da7013af4e0c3ddd321d76f93d"} Jun 09 14:13:26.241378 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:26.241365 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:13:26.254122 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:26.254077 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bcb8m" podStartSLOduration=65.386441044 podStartE2EDuration="1m10.254064757s" totalImagesPullingTime="4.867623713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:13:26.253373965 +0000 UTC m=+70.900120669" watchObservedRunningTime="2026-06-09 14:13:26.254064757 +0000 UTC m=+70.900811450" Jun 09 14:13:26.266471 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:26.266430 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lsq9c" podStartSLOduration=66.826365984 podStartE2EDuration="1m11.266420465s" totalImagesPullingTime="4.440054481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:13:26.265644331 +0000 UTC m=+70.912391036" watchObservedRunningTime="2026-06-09 14:13:26.266420465 +0000 UTC m=+70.913167150" Jun 09 14:13:54.499849 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:54.499803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:13:54.500273 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:54.499969 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:13:54.500273 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:13:54.500044 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:14:58.500027971 +0000 UTC m=+163.146774645 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:13:57.246155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:13:57.246123 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lsq9c" Jun 09 14:14:21.345585 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:14:21.345552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:14:24.600635 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:14:24.600575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:14:24.601080 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:14:24.600759 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 14:14:24.601080 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:14:24.600855 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs podName:13916a76-8368-4bc7-b6d0-55615e3aab9e nodeName:}" failed. No retries permitted until 2026-06-09 14:16:26.600836732 +0000 UTC m=+251.247583404 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs") pod "network-metrics-daemon-hsxpw" (UID: "13916a76-8368-4bc7-b6d0-55615e3aab9e") : secret "metrics-daemon-secret" not found Jun 09 14:14:53.710486 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:14:53.710424 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-54mcq" podUID="a06ae45d-d42d-4ed3-9770-95289acac343" Jun 09 14:14:53.887312 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:14:53.887273 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hsxpw" podUID="13916a76-8368-4bc7-b6d0-55615e3aab9e" Jun 09 14:14:54.443524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:14:54.443491 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54mcq" Jun 09 14:14:58.363998 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:14:58.363914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:14:58.529190 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:14:58.529145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:14:58.529364 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:14:58.529310 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 14:14:58.529429 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:14:58.529390 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls podName:a06ae45d-d42d-4ed3-9770-95289acac343 nodeName:}" failed. No retries permitted until 2026-06-09 14:17:00.529372083 +0000 UTC m=+285.176118754 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls") pod "dns-default-54mcq" (UID: "a06ae45d-d42d-4ed3-9770-95289acac343") : secret "dns-default-metrics-tls" not found Jun 09 14:15:05.872791 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:05.872742 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:15:52.344097 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.344059 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl"] Jun 09 14:15:52.346636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.346615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.349841 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.349820 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 09 14:15:52.349943 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.349843 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 09 14:15:52.349943 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.349854 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 09 14:15:52.349943 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.349843 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 09 14:15:52.349943 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.349854 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tpv5c\"" Jun 09 14:15:52.351488 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.351467 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-868bcbfc76-pjd4b"] Jun 09 14:15:52.354019 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.354000 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5"] Jun 09 14:15:52.354150 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.354132 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.356026 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.356009 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:52.356623 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.356607 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 09 14:15:52.356755 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.356738 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 09 14:15:52.356853 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.356839 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s5fz5\"" Jun 09 14:15:52.356919 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.356859 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 09 14:15:52.358553 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.358533 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 09 14:15:52.358671 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.358582 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pvskr\"" Jun 09 14:15:52.359347 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.359328 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl"] Jun 09 14:15:52.359742 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.359724 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 09 14:15:52.364975 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.364949 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 09 14:15:52.369021 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.368995 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5"] Jun 09 14:15:52.372018 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.371997 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868bcbfc76-pjd4b"] Jun 09 14:15:52.400413 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400374 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55cc31f5-c7f1-440b-92e1-ef3068e777e7-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.400580 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-image-registry-private-configuration\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400580 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400492 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-trusted-ca\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400579 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400647 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:52.400762 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ebca47e-d091-429d-a238-2583e0da2bd3-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:52.400849 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44896588-ab56-415a-a983-5aec7314129a-ca-trust-extracted\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400893 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-installation-pull-secrets\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400893 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-registry-certificates\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400957 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400904 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-bound-sa-token\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400957 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9vfp\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-kube-api-access-h9vfp\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.400957 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75f8j\" (UniqueName: \"kubernetes.io/projected/55cc31f5-c7f1-440b-92e1-ef3068e777e7-kube-api-access-75f8j\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.401048 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.400962 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55cc31f5-c7f1-440b-92e1-ef3068e777e7-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.442447 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.442391 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr"] Jun 09 14:15:52.444937 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.444918 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" Jun 09 14:15:52.446378 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.446348 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-hf2bv"] Jun 09 14:15:52.447580 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.447560 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 09 14:15:52.447685 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.447677 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5kkxm\"" Jun 09 14:15:52.448313 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.448300 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 09 14:15:52.448812 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.448796 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq"] Jun 09 14:15:52.448913 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.448900 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.450967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.450948 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 14:15:52.451083 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.451025 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq"] Jun 09 14:15:52.451194 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.451173 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.451255 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.451212 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 14:15:52.451429 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.451386 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 09 14:15:52.451974 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.451952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mzb82\"" Jun 09 14:15:52.453376 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.453358 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" Jun 09 14:15:52.454237 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.454220 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 14:15:52.454413 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.454383 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 14:15:52.454833 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.454819 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gcq2g\"" Jun 09 14:15:52.454889 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.454841 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 09 14:15:52.454942 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.454847 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 09 14:15:52.454942 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.454927 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 09 14:15:52.455535 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.455519 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-h6pj7\"" Jun 09 14:15:52.457109 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.457090 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr"] Jun 09 14:15:52.457458 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.457439 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 09 14:15:52.457573 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.457552 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:52.460762 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.460736 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-hf2bv"] Jun 09 14:15:52.464855 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.464838 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq"] Jun 09 14:15:52.476818 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.476747 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq"] Jun 09 14:15:52.501335 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501308 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jttgb\" (UniqueName: \"kubernetes.io/projected/023aaede-6273-4afe-8d49-115a2585690b-kube-api-access-jttgb\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.501335 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501337 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glpkn\" (UniqueName: \"kubernetes.io/projected/ef8f335f-ed20-433c-85ad-b8199bee7c60-kube-api-access-glpkn\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.501540 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55cc31f5-c7f1-440b-92e1-ef3068e777e7-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.501540 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501416 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/023aaede-6273-4afe-8d49-115a2585690b-trusted-ca-bundle\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.501540 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501436 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-image-registry-private-configuration\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.501540 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wsvt\" (UniqueName: \"kubernetes.io/projected/02143712-146d-41ab-9b7d-b1da35dce3bf-kube-api-access-4wsvt\") pod \"volume-data-source-validator-78dd696465-8xkfr\" (UID: \"02143712-146d-41ab-9b7d-b1da35dce3bf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" Jun 09 14:15:52.501540 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-registry-certificates\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.501540 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501518 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/023aaede-6273-4afe-8d49-115a2585690b-snapshots\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.501846 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75f8j\" (UniqueName: \"kubernetes.io/projected/55cc31f5-c7f1-440b-92e1-ef3068e777e7-kube-api-access-75f8j\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.501846 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/023aaede-6273-4afe-8d49-115a2585690b-serving-cert\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.501846 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:52.501846 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-installation-pull-secrets\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.501846 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/023aaede-6273-4afe-8d49-115a2585690b-service-ca-bundle\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.502103 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.502103 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.501897 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 14:15:52.502103 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.501903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbwz7\" (UniqueName: \"kubernetes.io/projected/1f3ea233-7e6c-441d-a918-d9145ac27ff3-kube-api-access-lbwz7\") pod \"network-check-source-dbd9cf844-5rvxq\" (UID: \"1f3ea233-7e6c-441d-a918-d9145ac27ff3\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" Jun 09 14:15:52.502103 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.501969 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert podName:8ebca47e-d091-429d-a238-2583e0da2bd3 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:53.001947168 +0000 UTC m=+217.648693840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6g4j5" (UID: "8ebca47e-d091-429d-a238-2583e0da2bd3") : secret "networking-console-plugin-cert" not found Jun 09 14:15:52.502103 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9vfp\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-kube-api-access-h9vfp\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.502103 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55cc31f5-c7f1-440b-92e1-ef3068e777e7-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.502432 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-trusted-ca\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.502432 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502135 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-registry-certificates\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.502432 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ef8f335f-ed20-433c-85ad-b8199bee7c60-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.502432 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.502645 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-bound-sa-token\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.502645 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502472 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/023aaede-6273-4afe-8d49-115a2585690b-tmp\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.502645 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ebca47e-d091-429d-a238-2583e0da2bd3-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:52.502645 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502523 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44896588-ab56-415a-a983-5aec7314129a-ca-trust-extracted\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.502645 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502631 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/55cc31f5-c7f1-440b-92e1-ef3068e777e7-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.502886 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.502681 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 14:15:52.502886 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.502698 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868bcbfc76-pjd4b: secret "image-registry-tls" not found Jun 09 14:15:52.502886 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.502755 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls podName:44896588-ab56-415a-a983-5aec7314129a nodeName:}" failed. No retries permitted until 2026-06-09 14:15:53.002736926 +0000 UTC m=+217.649483602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls") pod "image-registry-868bcbfc76-pjd4b" (UID: "44896588-ab56-415a-a983-5aec7314129a") : secret "image-registry-tls" not found Jun 09 14:15:52.502886 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.502861 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44896588-ab56-415a-a983-5aec7314129a-ca-trust-extracted\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.503238 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.503217 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ebca47e-d091-429d-a238-2583e0da2bd3-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:52.503936 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.503915 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-trusted-ca\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.504139 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.504120 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55cc31f5-c7f1-440b-92e1-ef3068e777e7-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.504241 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.504223 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-installation-pull-secrets\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.504585 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.504567 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-image-registry-private-configuration\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.512594 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.512572 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-bound-sa-token\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.512949 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.512928 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9vfp\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-kube-api-access-h9vfp\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:52.513090 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.513072 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75f8j\" (UniqueName: \"kubernetes.io/projected/55cc31f5-c7f1-440b-92e1-ef3068e777e7-kube-api-access-75f8j\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-4m8dl\" (UID: \"55cc31f5-c7f1-440b-92e1-ef3068e777e7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.524337 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.524304 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:52.603776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603679 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jttgb\" (UniqueName: \"kubernetes.io/projected/023aaede-6273-4afe-8d49-115a2585690b-kube-api-access-jttgb\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.603776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603717 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glpkn\" (UniqueName: \"kubernetes.io/projected/ef8f335f-ed20-433c-85ad-b8199bee7c60-kube-api-access-glpkn\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.603776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/023aaede-6273-4afe-8d49-115a2585690b-trusted-ca-bundle\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603780 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wsvt\" (UniqueName: \"kubernetes.io/projected/02143712-146d-41ab-9b7d-b1da35dce3bf-kube-api-access-4wsvt\") pod \"volume-data-source-validator-78dd696465-8xkfr\" (UID: \"02143712-146d-41ab-9b7d-b1da35dce3bf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/023aaede-6273-4afe-8d49-115a2585690b-snapshots\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/023aaede-6273-4afe-8d49-115a2585690b-serving-cert\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603873 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/023aaede-6273-4afe-8d49-115a2585690b-service-ca-bundle\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603900 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.603928 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbwz7\" (UniqueName: \"kubernetes.io/projected/1f3ea233-7e6c-441d-a918-d9145ac27ff3-kube-api-access-lbwz7\") pod \"network-check-source-dbd9cf844-5rvxq\" (UID: \"1f3ea233-7e6c-441d-a918-d9145ac27ff3\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" Jun 09 14:15:52.604057 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ef8f335f-ed20-433c-85ad-b8199bee7c60-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.604352 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/023aaede-6273-4afe-8d49-115a2585690b-tmp\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604352 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.604153 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:52.604352 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:52.604226 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls podName:ef8f335f-ed20-433c-85ad-b8199bee7c60 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:53.104206217 +0000 UTC m=+217.750952905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-52gfq" (UID: "ef8f335f-ed20-433c-85ad-b8199bee7c60") : secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:52.604554 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604479 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/023aaede-6273-4afe-8d49-115a2585690b-service-ca-bundle\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604554 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604524 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/023aaede-6273-4afe-8d49-115a2585690b-snapshots\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604698 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604653 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/023aaede-6273-4afe-8d49-115a2585690b-tmp\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604837 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604812 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/023aaede-6273-4afe-8d49-115a2585690b-trusted-ca-bundle\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.604920 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.604905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ef8f335f-ed20-433c-85ad-b8199bee7c60-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.606211 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.606187 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:52.606707 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.606692 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/023aaede-6273-4afe-8d49-115a2585690b-serving-cert\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.612395 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.612364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glpkn\" (UniqueName: \"kubernetes.io/projected/ef8f335f-ed20-433c-85ad-b8199bee7c60-kube-api-access-glpkn\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:52.612663 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.612641 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jttgb\" (UniqueName: \"kubernetes.io/projected/023aaede-6273-4afe-8d49-115a2585690b-kube-api-access-jttgb\") pod \"insights-operator-896549976-hf2bv\" (UID: \"023aaede-6273-4afe-8d49-115a2585690b\") " pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.612739 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.612720 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbwz7\" (UniqueName: \"kubernetes.io/projected/1f3ea233-7e6c-441d-a918-d9145ac27ff3-kube-api-access-lbwz7\") pod \"network-check-source-dbd9cf844-5rvxq\" (UID: \"1f3ea233-7e6c-441d-a918-d9145ac27ff3\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" Jun 09 14:15:52.613072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.613053 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wsvt\" (UniqueName: \"kubernetes.io/projected/02143712-146d-41ab-9b7d-b1da35dce3bf-kube-api-access-4wsvt\") pod \"volume-data-source-validator-78dd696465-8xkfr\" (UID: \"02143712-146d-41ab-9b7d-b1da35dce3bf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" Jun 09 14:15:52.655965 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.655932 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" Jun 09 14:15:52.707722 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.707643 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:52.755660 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.755618 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" Jun 09 14:15:52.764870 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.764844 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-hf2bv" Jun 09 14:15:52.775255 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.775234 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" Jun 09 14:15:52.813196 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.813160 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl"] Jun 09 14:15:52.818970 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:15:52.818572 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cc31f5_c7f1_440b_92e1_ef3068e777e7.slice/crio-b215a7644727761f790431ebc4e89510cd440f676e8b4dd7be56ad42ad57dbff WatchSource:0}: Error finding container b215a7644727761f790431ebc4e89510cd440f676e8b4dd7be56ad42ad57dbff: Status 404 returned error can't find the container with id b215a7644727761f790431ebc4e89510cd440f676e8b4dd7be56ad42ad57dbff Jun 09 14:15:52.909003 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.908057 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr"] Jun 09 14:15:52.913387 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:15:52.913352 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02143712_146d_41ab_9b7d_b1da35dce3bf.slice/crio-1460f678f8efd860baeab593788de9a10e715256cfdc6ac5f3f40fec2156e8e6 WatchSource:0}: Error finding container 1460f678f8efd860baeab593788de9a10e715256cfdc6ac5f3f40fec2156e8e6: Status 404 returned error can't find the container with id 1460f678f8efd860baeab593788de9a10e715256cfdc6ac5f3f40fec2156e8e6 Jun 09 14:15:52.927037 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.927009 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-hf2bv"] Jun 09 14:15:52.929769 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:15:52.929746 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023aaede_6273_4afe_8d49_115a2585690b.slice/crio-8fa91c781a34a41232626f623109d74aa8c41b4ce20655dbfeaaa3824ad913e4 WatchSource:0}: Error finding container 8fa91c781a34a41232626f623109d74aa8c41b4ce20655dbfeaaa3824ad913e4: Status 404 returned error can't find the container with id 8fa91c781a34a41232626f623109d74aa8c41b4ce20655dbfeaaa3824ad913e4 Jun 09 14:15:52.940055 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:52.940034 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq"] Jun 09 14:15:52.942606 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:15:52.942586 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3ea233_7e6c_441d_a918_d9145ac27ff3.slice/crio-655f59ceb413b5d5916f047547ad7733f3ba703b5d29fed7bc0db94cc2a5c676 WatchSource:0}: Error finding container 655f59ceb413b5d5916f047547ad7733f3ba703b5d29fed7bc0db94cc2a5c676: Status 404 returned error can't find the container with id 655f59ceb413b5d5916f047547ad7733f3ba703b5d29fed7bc0db94cc2a5c676 Jun 09 14:15:53.007530 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.007504 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:53.007673 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.007562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:53.007673 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.007658 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 14:15:53.007673 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.007665 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 14:15:53.007785 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.007679 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868bcbfc76-pjd4b: secret "image-registry-tls" not found Jun 09 14:15:53.007785 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.007716 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert podName:8ebca47e-d091-429d-a238-2583e0da2bd3 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:54.007699779 +0000 UTC m=+218.654446449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6g4j5" (UID: "8ebca47e-d091-429d-a238-2583e0da2bd3") : secret "networking-console-plugin-cert" not found Jun 09 14:15:53.007785 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.007733 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls podName:44896588-ab56-415a-a983-5aec7314129a nodeName:}" failed. No retries permitted until 2026-06-09 14:15:54.00772416 +0000 UTC m=+218.654470831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls") pod "image-registry-868bcbfc76-pjd4b" (UID: "44896588-ab56-415a-a983-5aec7314129a") : secret "image-registry-tls" not found Jun 09 14:15:53.108616 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.108573 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:53.108821 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.108745 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:53.108890 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:53.108831 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls podName:ef8f335f-ed20-433c-85ad-b8199bee7c60 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:54.10881036 +0000 UTC m=+218.755557046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-52gfq" (UID: "ef8f335f-ed20-433c-85ad-b8199bee7c60") : secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:53.582137 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.582029 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" event={"ID":"02143712-146d-41ab-9b7d-b1da35dce3bf","Type":"ContainerStarted","Data":"1460f678f8efd860baeab593788de9a10e715256cfdc6ac5f3f40fec2156e8e6"} Jun 09 14:15:53.583806 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.583728 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hf2bv" event={"ID":"023aaede-6273-4afe-8d49-115a2585690b","Type":"ContainerStarted","Data":"8fa91c781a34a41232626f623109d74aa8c41b4ce20655dbfeaaa3824ad913e4"} Jun 09 14:15:53.586147 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.586080 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" event={"ID":"55cc31f5-c7f1-440b-92e1-ef3068e777e7","Type":"ContainerStarted","Data":"b215a7644727761f790431ebc4e89510cd440f676e8b4dd7be56ad42ad57dbff"} Jun 09 14:15:53.589881 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.589794 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" event={"ID":"1f3ea233-7e6c-441d-a918-d9145ac27ff3","Type":"ContainerStarted","Data":"9415913af0d93fb239d6053d26f5658c907fd3897191e25b0a0e93252de9e984"} Jun 09 14:15:53.589881 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.589836 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" event={"ID":"1f3ea233-7e6c-441d-a918-d9145ac27ff3","Type":"ContainerStarted","Data":"655f59ceb413b5d5916f047547ad7733f3ba703b5d29fed7bc0db94cc2a5c676"} Jun 09 14:15:53.607149 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:53.606430 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-5rvxq" podStartSLOduration=1.606395684 podStartE2EDuration="1.606395684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:15:53.605517232 +0000 UTC m=+218.252263925" watchObservedRunningTime="2026-06-09 14:15:53.606395684 +0000 UTC m=+218.253142378" Jun 09 14:15:54.015463 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:54.015418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:54.015677 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:54.015518 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:54.015677 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.015552 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 14:15:54.015677 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.015633 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert podName:8ebca47e-d091-429d-a238-2583e0da2bd3 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:56.015610608 +0000 UTC m=+220.662357324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6g4j5" (UID: "8ebca47e-d091-429d-a238-2583e0da2bd3") : secret "networking-console-plugin-cert" not found Jun 09 14:15:54.015677 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.015654 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 14:15:54.015677 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.015668 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868bcbfc76-pjd4b: secret "image-registry-tls" not found Jun 09 14:15:54.015931 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.015703 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls podName:44896588-ab56-415a-a983-5aec7314129a nodeName:}" failed. No retries permitted until 2026-06-09 14:15:56.015692221 +0000 UTC m=+220.662438894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls") pod "image-registry-868bcbfc76-pjd4b" (UID: "44896588-ab56-415a-a983-5aec7314129a") : secret "image-registry-tls" not found Jun 09 14:15:54.116710 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:54.116677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:54.116906 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.116877 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:54.117052 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:54.116973 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls podName:ef8f335f-ed20-433c-85ad-b8199bee7c60 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:56.116948293 +0000 UTC m=+220.763694964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-52gfq" (UID: "ef8f335f-ed20-433c-85ad-b8199bee7c60") : secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:54.597491 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:54.597446 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" event={"ID":"02143712-146d-41ab-9b7d-b1da35dce3bf","Type":"ContainerStarted","Data":"94bd40856a8a94b55f581e238bd0ec6c8dc6e6752d8bd5d9ca2bfc698db94efa"} Jun 09 14:15:54.612525 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:54.612474 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-8xkfr" podStartSLOduration=1.353421159 podStartE2EDuration="2.612456115s" totalImagesPullingTime="1.259034956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:15:54.611781892 +0000 UTC m=+219.258528587" watchObservedRunningTime="2026-06-09 14:15:54.612456115 +0000 UTC m=+219.259202809" Jun 09 14:15:54.694457 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:54.694418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:55.603036 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:55.602945 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hf2bv" event={"ID":"023aaede-6273-4afe-8d49-115a2585690b","Type":"ContainerStarted","Data":"0af5925b9a2f4b366c852516fc2a7719204774934f95beb7d5fe81ee00a8feae"} Jun 09 14:15:55.604521 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:55.604494 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" event={"ID":"55cc31f5-c7f1-440b-92e1-ef3068e777e7","Type":"ContainerStarted","Data":"97166b126959d06c08fa5ac1ca33e18de90daaaf635dcbf4ca23dc17a25e361d"} Jun 09 14:15:55.618521 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:55.618476 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-hf2bv" podStartSLOduration=1.296686287 podStartE2EDuration="3.618457995s" totalImagesPullingTime="2.321771708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:15:55.617584965 +0000 UTC m=+220.264331657" watchObservedRunningTime="2026-06-09 14:15:55.618457995 +0000 UTC m=+220.265204683" Jun 09 14:15:55.633211 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:55.633166 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" podStartSLOduration=1.201410688 podStartE2EDuration="3.633152114s" totalImagesPullingTime="2.431741426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:15:55.632391841 +0000 UTC m=+220.279138536" watchObservedRunningTime="2026-06-09 14:15:55.633152114 +0000 UTC m=+220.279898806" Jun 09 14:15:56.036014 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.035964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:15:56.036222 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.036068 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:15:56.036222 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.036113 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 14:15:56.036222 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.036181 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert podName:8ebca47e-d091-429d-a238-2583e0da2bd3 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:00.036165254 +0000 UTC m=+224.682911925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6g4j5" (UID: "8ebca47e-d091-429d-a238-2583e0da2bd3") : secret "networking-console-plugin-cert" not found Jun 09 14:15:56.036222 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.036198 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 14:15:56.036222 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.036211 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868bcbfc76-pjd4b: secret "image-registry-tls" not found Jun 09 14:15:56.036513 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.036257 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls podName:44896588-ab56-415a-a983-5aec7314129a nodeName:}" failed. No retries permitted until 2026-06-09 14:16:00.036241973 +0000 UTC m=+224.682988668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls") pod "image-registry-868bcbfc76-pjd4b" (UID: "44896588-ab56-415a-a983-5aec7314129a") : secret "image-registry-tls" not found Jun 09 14:15:56.136640 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.136602 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:15:56.136838 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.136738 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:56.136838 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:56.136807 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls podName:ef8f335f-ed20-433c-85ad-b8199bee7c60 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:00.136790707 +0000 UTC m=+224.783537397 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-52gfq" (UID: "ef8f335f-ed20-433c-85ad-b8199bee7c60") : secret "cluster-monitoring-operator-tls" not found Jun 09 14:15:56.697369 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.697318 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:56.699812 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.699788 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68"] Jun 09 14:15:56.701973 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.701959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" Jun 09 14:15:56.704201 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.704181 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 09 14:15:56.704952 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.704927 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ntrfp\"" Jun 09 14:15:56.705060 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.705000 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 09 14:15:56.711838 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.711819 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68"] Jun 09 14:15:56.741733 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.741691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qsbn\" (UniqueName: \"kubernetes.io/projected/e9604d2b-81fc-4a1e-b664-ecd369cf2854-kube-api-access-5qsbn\") pod \"migrator-955ffdf5f-mvc68\" (UID: \"e9604d2b-81fc-4a1e-b664-ecd369cf2854\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" Jun 09 14:15:56.789897 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.789867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:56.842126 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.842094 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qsbn\" (UniqueName: \"kubernetes.io/projected/e9604d2b-81fc-4a1e-b664-ecd369cf2854-kube-api-access-5qsbn\") pod \"migrator-955ffdf5f-mvc68\" (UID: \"e9604d2b-81fc-4a1e-b664-ecd369cf2854\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" Jun 09 14:15:56.849594 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.849573 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qsbn\" (UniqueName: \"kubernetes.io/projected/e9604d2b-81fc-4a1e-b664-ecd369cf2854-kube-api-access-5qsbn\") pod \"migrator-955ffdf5f-mvc68\" (UID: \"e9604d2b-81fc-4a1e-b664-ecd369cf2854\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" Jun 09 14:15:56.895147 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:56.895113 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:57.012155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.012111 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" Jun 09 14:15:57.134909 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.134881 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68"] Jun 09 14:15:57.137866 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:15:57.137841 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9604d2b_81fc_4a1e_b664_ecd369cf2854.slice/crio-7877ded12f07e56e20cc813f6a6638c3828110cc5fc45e3e25899fbe3f72c382 WatchSource:0}: Error finding container 7877ded12f07e56e20cc813f6a6638c3828110cc5fc45e3e25899fbe3f72c382: Status 404 returned error can't find the container with id 7877ded12f07e56e20cc813f6a6638c3828110cc5fc45e3e25899fbe3f72c382 Jun 09 14:15:57.267855 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.267758 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:57.320092 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.320054 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:57.490798 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.490766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:57.542047 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.541964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:57.612328 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.612300 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" event={"ID":"e9604d2b-81fc-4a1e-b664-ecd369cf2854","Type":"ContainerStarted","Data":"7877ded12f07e56e20cc813f6a6638c3828110cc5fc45e3e25899fbe3f72c382"} Jun 09 14:15:57.975646 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:57.975560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:58.076202 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.076164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:58.573783 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.573699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:58.616637 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.616602 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" event={"ID":"e9604d2b-81fc-4a1e-b664-ecd369cf2854","Type":"ContainerStarted","Data":"407f049e841264eb4a124a051440e45930fc7779cd3591600865868818197aa8"} Jun 09 14:15:58.616800 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.616663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" event={"ID":"e9604d2b-81fc-4a1e-b664-ecd369cf2854","Type":"ContainerStarted","Data":"20d3e2baf1c39d0521b1c18e13f9bfd2eb73b9571e533ab849532499a3237c1e"} Jun 09 14:15:58.632013 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.631956 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-mvc68" podStartSLOduration=1.586306277 podStartE2EDuration="2.631938605s" totalImagesPullingTime="1.045632328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:15:58.631835886 +0000 UTC m=+223.278582579" watchObservedRunningTime="2026-06-09 14:15:58.631938605 +0000 UTC m=+223.278685298" Jun 09 14:15:58.675623 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.675592 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fl4nq"] Jun 09 14:15:58.678419 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.678379 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.681094 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.681070 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 14:15:58.681215 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.681117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7cblk\"" Jun 09 14:15:58.681215 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.681076 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 14:15:58.688027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.688004 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fl4nq"] Jun 09 14:15:58.715885 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.715856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:58.756681 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.756650 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l44fd\" (UniqueName: \"kubernetes.io/projected/74529d19-d865-4dba-a40e-b7e63494eed2-kube-api-access-l44fd\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.756829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.756704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74529d19-d865-4dba-a40e-b7e63494eed2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.756829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.756797 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/74529d19-d865-4dba-a40e-b7e63494eed2-data-volume\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.756829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.756822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.756949 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.756839 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/74529d19-d865-4dba-a40e-b7e63494eed2-crio-socket\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.830291 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.830228 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rk89k_8da21813-284f-4169-b06c-dd9b636cdc8f/dns-node-resolver/0.log" Jun 09 14:15:58.857689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.857657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l44fd\" (UniqueName: \"kubernetes.io/projected/74529d19-d865-4dba-a40e-b7e63494eed2-kube-api-access-l44fd\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.857823 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.857725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74529d19-d865-4dba-a40e-b7e63494eed2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.857823 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.857809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/74529d19-d865-4dba-a40e-b7e63494eed2-data-volume\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.857905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.857832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.857905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.857855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/74529d19-d865-4dba-a40e-b7e63494eed2-crio-socket\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.857982 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.857933 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/74529d19-d865-4dba-a40e-b7e63494eed2-crio-socket\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.858017 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:58.857973 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 09 14:15:58.858068 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:58.858053 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls podName:74529d19-d865-4dba-a40e-b7e63494eed2 nodeName:}" failed. No retries permitted until 2026-06-09 14:15:59.358030929 +0000 UTC m=+224.004777621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl4nq" (UID: "74529d19-d865-4dba-a40e-b7e63494eed2") : secret "insights-runtime-extractor-tls" not found Jun 09 14:15:58.858165 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.858146 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/74529d19-d865-4dba-a40e-b7e63494eed2-data-volume\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.858246 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.858231 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/74529d19-d865-4dba-a40e-b7e63494eed2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:58.866251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:58.866222 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l44fd\" (UniqueName: \"kubernetes.io/projected/74529d19-d865-4dba-a40e-b7e63494eed2-kube-api-access-l44fd\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:59.363147 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:59.363100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:15:59.363579 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:59.363254 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 09 14:15:59.363579 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:15:59.363322 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls podName:74529d19-d865-4dba-a40e-b7e63494eed2 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:00.363306169 +0000 UTC m=+225.010052839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl4nq" (UID: "74529d19-d865-4dba-a40e-b7e63494eed2") : secret "insights-runtime-extractor-tls" not found Jun 09 14:15:59.429644 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:59.429615 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vvsn4_a18c19c3-e11d-4191-81a4-73707986bee7/node-ca/0.log" Jun 09 14:15:59.677019 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:59.676930 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:15:59.774635 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:15:59.774609 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:00.068242 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:00.068204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:00.068474 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:00.068281 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:16:00.068474 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.068386 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 14:16:00.068474 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.068428 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 14:16:00.068474 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.068434 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868bcbfc76-pjd4b: secret "image-registry-tls" not found Jun 09 14:16:00.068643 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.068496 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls podName:44896588-ab56-415a-a983-5aec7314129a nodeName:}" failed. No retries permitted until 2026-06-09 14:16:08.068472348 +0000 UTC m=+232.715219019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls") pod "image-registry-868bcbfc76-pjd4b" (UID: "44896588-ab56-415a-a983-5aec7314129a") : secret "image-registry-tls" not found Jun 09 14:16:00.068643 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.068515 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert podName:8ebca47e-d091-429d-a238-2583e0da2bd3 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:08.068505683 +0000 UTC m=+232.715252355 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6g4j5" (UID: "8ebca47e-d091-429d-a238-2583e0da2bd3") : secret "networking-console-plugin-cert" not found Jun 09 14:16:00.168900 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:00.168865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:16:00.169058 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.169039 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 09 14:16:00.169135 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.169121 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls podName:ef8f335f-ed20-433c-85ad-b8199bee7c60 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:08.169100472 +0000 UTC m=+232.815847160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-52gfq" (UID: "ef8f335f-ed20-433c-85ad-b8199bee7c60") : secret "cluster-monitoring-operator-tls" not found Jun 09 14:16:00.370094 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:00.369992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:16:00.370519 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.370146 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 09 14:16:00.370519 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:00.370211 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls podName:74529d19-d865-4dba-a40e-b7e63494eed2 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:02.370196154 +0000 UTC m=+227.016942824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl4nq" (UID: "74529d19-d865-4dba-a40e-b7e63494eed2") : secret "insights-runtime-extractor-tls" not found Jun 09 14:16:00.431096 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:00.431069 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hvcpz_d6221db5-4346-494c-85c3-9cf89b8cba98/serve-healthcheck-canary/0.log" Jun 09 14:16:00.497899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:00.497861 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:01.013237 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:01.013197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:01.179592 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:01.179556 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:01.279684 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:01.279600 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:01.379810 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:01.379776 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:01.712569 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:01.712476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:02.386705 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:02.386670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:16:02.387035 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:02.386824 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 09 14:16:02.387035 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:02.386890 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls podName:74529d19-d865-4dba-a40e-b7e63494eed2 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:06.386873536 +0000 UTC m=+231.033620207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl4nq" (UID: "74529d19-d865-4dba-a40e-b7e63494eed2") : secret "insights-runtime-extractor-tls" not found Jun 09 14:16:02.710504 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:02.710417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:03.115871 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:03.115835 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:03.311091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:03.311050 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:03.366857 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:03.366778 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:03.517144 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:03.517106 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:03.714124 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:03.714031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:04.714723 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:04.714693 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:04.814943 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:04.814912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:04.916893 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:04.916865 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:05.109511 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:05.109483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:05.263553 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:05.263518 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:05.364145 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:05.364064 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:05.505044 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:05.505015 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:05.914555 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:05.914517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:06.422366 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:06.422330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:16:06.422562 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:06.422511 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 09 14:16:06.422606 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:06.422597 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls podName:74529d19-d865-4dba-a40e-b7e63494eed2 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:14.4225779 +0000 UTC m=+239.069324591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl4nq" (UID: "74529d19-d865-4dba-a40e-b7e63494eed2") : secret "insights-runtime-extractor-tls" not found Jun 09 14:16:06.508709 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:06.508672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:06.833674 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:06.833637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:07.110861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:07.110775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:07.308422 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:07.308375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:07.530317 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:07.530285 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:07.629811 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:07.629779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:07.916933 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:07.916849 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:08.136674 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.136636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:16:08.137114 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.136730 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:08.137114 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:08.136797 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 14:16:08.137114 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:08.136865 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert podName:8ebca47e-d091-429d-a238-2583e0da2bd3 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:24.136849544 +0000 UTC m=+248.783596215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6g4j5" (UID: "8ebca47e-d091-429d-a238-2583e0da2bd3") : secret "networking-console-plugin-cert" not found Jun 09 14:16:08.139346 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.139315 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"image-registry-868bcbfc76-pjd4b\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:08.237517 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.237486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:16:08.237685 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:08.237643 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 09 14:16:08.237736 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:08.237725 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls podName:ef8f335f-ed20-433c-85ad-b8199bee7c60 nodeName:}" failed. No retries permitted until 2026-06-09 14:16:24.237707736 +0000 UTC m=+248.884454407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-52gfq" (UID: "ef8f335f-ed20-433c-85ad-b8199bee7c60") : secret "cluster-monitoring-operator-tls" not found Jun 09 14:16:08.266609 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.266573 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:08.394904 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.394716 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868bcbfc76-pjd4b"] Jun 09 14:16:08.397394 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:08.397369 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44896588_ab56_415a_a983_5aec7314129a.slice/crio-10466f636675757e7cf74cd9d4892d9f3540931c3f65335563f963069ac196dc WatchSource:0}: Error finding container 10466f636675757e7cf74cd9d4892d9f3540931c3f65335563f963069ac196dc: Status 404 returned error can't find the container with id 10466f636675757e7cf74cd9d4892d9f3540931c3f65335563f963069ac196dc Jun 09 14:16:08.513309 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.513224 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:08.649530 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.649495 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" event={"ID":"44896588-ab56-415a-a983-5aec7314129a","Type":"ContainerStarted","Data":"f3b4828fd884db63a74d58f3ec2f6224b0f3d414b46ef8c68dd6f4187ea82229"} Jun 09 14:16:08.649696 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.649543 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" event={"ID":"44896588-ab56-415a-a983-5aec7314129a","Type":"ContainerStarted","Data":"10466f636675757e7cf74cd9d4892d9f3540931c3f65335563f963069ac196dc"} Jun 09 14:16:08.649696 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.649623 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:08.666101 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.666044 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" podStartSLOduration=16.666029396 podStartE2EDuration="16.666029396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:08.66573298 +0000 UTC m=+233.312479673" watchObservedRunningTime="2026-06-09 14:16:08.666029396 +0000 UTC m=+233.312776067" Jun 09 14:16:08.907467 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:08.907369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:09.310988 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:09.310953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:09.508654 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:09.508620 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:10.111592 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:10.111557 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:10.511911 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:10.511881 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:10.907939 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:10.907855 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:11.107213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:11.107177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:11.507766 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:11.507730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:11.712865 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:11.712830 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:14.494338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:14.494280 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:16:14.496699 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:14.496678 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/74529d19-d865-4dba-a40e-b7e63494eed2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl4nq\" (UID: \"74529d19-d865-4dba-a40e-b7e63494eed2\") " pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:16:14.587291 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:14.587241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fl4nq" Jun 09 14:16:14.715070 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:14.715038 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fl4nq"] Jun 09 14:16:14.718327 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:14.718301 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74529d19_d865_4dba_a40e_b7e63494eed2.slice/crio-96613e7165c16b2741ec6ee1c0ac611d0927311854cc2c7606269e6177d8cbc4 WatchSource:0}: Error finding container 96613e7165c16b2741ec6ee1c0ac611d0927311854cc2c7606269e6177d8cbc4: Status 404 returned error can't find the container with id 96613e7165c16b2741ec6ee1c0ac611d0927311854cc2c7606269e6177d8cbc4 Jun 09 14:16:15.645131 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:15.645041 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:15.675158 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:15.675123 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl4nq" event={"ID":"74529d19-d865-4dba-a40e-b7e63494eed2","Type":"ContainerStarted","Data":"4bbb13ee2222e3972debbec6b3dce6285e09f82c18c096c6ec6c77e6ba4fbab0"} Jun 09 14:16:15.675297 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:15.675184 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl4nq" event={"ID":"74529d19-d865-4dba-a40e-b7e63494eed2","Type":"ContainerStarted","Data":"139b3888cb1c6aadbf5abb65f0efb1f8cba9c20f5c405ea6a82c25ea6a6bce97"} Jun 09 14:16:15.675297 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:15.675210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl4nq" event={"ID":"74529d19-d865-4dba-a40e-b7e63494eed2","Type":"ContainerStarted","Data":"96613e7165c16b2741ec6ee1c0ac611d0927311854cc2c7606269e6177d8cbc4"} Jun 09 14:16:15.703969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:15.703939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:15.769969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:15.769940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:16.004373 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:16.004330 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:16.335315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:16.335231 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:16.723945 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:16.723878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:16.824324 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:16.824298 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:17.348056 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:17.348016 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:17.402409 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:17.402377 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:17.538591 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:17.538541 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:17.638796 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:17.638704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:17.684495 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:17.684458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl4nq" event={"ID":"74529d19-d865-4dba-a40e-b7e63494eed2","Type":"ContainerStarted","Data":"cc35aa810187520d92ed2604184df9b8b134ee5bb49a2a8146188ef5fd9f4486"} Jun 09 14:16:17.702751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:17.702674 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fl4nq" podStartSLOduration=17.786463382 podStartE2EDuration="19.702658493s" totalImagesPullingTime="1.916195111s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:15:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:17.700674453 +0000 UTC m=+242.347421169" watchObservedRunningTime="2026-06-09 14:16:17.702658493 +0000 UTC m=+242.349405207" Jun 09 14:16:20.268992 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:20.268954 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:20.369898 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:20.369868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:22.712739 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.712700 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-868bcbfc76-pjd4b"] Jun 09 14:16:22.723509 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.723477 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-8swkf"] Jun 09 14:16:22.729042 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.729027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:22.731606 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.731587 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cmhtp\"" Jun 09 14:16:22.731606 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.731603 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 09 14:16:22.731879 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.731864 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 09 14:16:22.738279 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.738256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-8swkf"] Jun 09 14:16:22.750898 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.750871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:22.821567 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.821536 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5499bf57fb-7fq4w"] Jun 09 14:16:22.823683 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.823666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.827559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.827537 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:22.841622 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.841598 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5499bf57fb-7fq4w"] Jun 09 14:16:22.860033 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.860006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxtg4\" (UniqueName: \"kubernetes.io/projected/eeca0437-0985-4c2b-907b-08d00f3fc0bd-kube-api-access-wxtg4\") pod \"downloads-6657875f97-8swkf\" (UID: \"eeca0437-0985-4c2b-907b-08d00f3fc0bd\") " pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:22.900547 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.900498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:22.961191 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961165 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f61f8351-2bf3-4012-82ea-77ee5fa864a0-ca-trust-extracted\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961201 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f61f8351-2bf3-4012-82ea-77ee5fa864a0-image-registry-private-configuration\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-registry-tls\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxtg4\" (UniqueName: \"kubernetes.io/projected/eeca0437-0985-4c2b-907b-08d00f3fc0bd-kube-api-access-wxtg4\") pod \"downloads-6657875f97-8swkf\" (UID: \"eeca0437-0985-4c2b-907b-08d00f3fc0bd\") " pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:22.961351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961273 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f61f8351-2bf3-4012-82ea-77ee5fa864a0-registry-certificates\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-bound-sa-token\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961637 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f61f8351-2bf3-4012-82ea-77ee5fa864a0-trusted-ca\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961637 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f61f8351-2bf3-4012-82ea-77ee5fa864a0-installation-pull-secrets\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.961637 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.961534 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mmxn\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-kube-api-access-5mmxn\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:22.970009 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:22.969945 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxtg4\" (UniqueName: \"kubernetes.io/projected/eeca0437-0985-4c2b-907b-08d00f3fc0bd-kube-api-access-wxtg4\") pod \"downloads-6657875f97-8swkf\" (UID: \"eeca0437-0985-4c2b-907b-08d00f3fc0bd\") " pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:23.038587 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.038542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:23.062179 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062151 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-bound-sa-token\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062207 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f61f8351-2bf3-4012-82ea-77ee5fa864a0-trusted-ca\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f61f8351-2bf3-4012-82ea-77ee5fa864a0-installation-pull-secrets\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062464 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mmxn\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-kube-api-access-5mmxn\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062464 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062347 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f61f8351-2bf3-4012-82ea-77ee5fa864a0-ca-trust-extracted\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062464 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f61f8351-2bf3-4012-82ea-77ee5fa864a0-image-registry-private-configuration\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062464 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-registry-tls\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062473 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f61f8351-2bf3-4012-82ea-77ee5fa864a0-registry-certificates\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.062887 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.062862 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f61f8351-2bf3-4012-82ea-77ee5fa864a0-ca-trust-extracted\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.063325 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.063252 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f61f8351-2bf3-4012-82ea-77ee5fa864a0-registry-certificates\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.063602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.063578 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f61f8351-2bf3-4012-82ea-77ee5fa864a0-trusted-ca\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.065002 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.064979 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f61f8351-2bf3-4012-82ea-77ee5fa864a0-installation-pull-secrets\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.065186 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.065165 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f61f8351-2bf3-4012-82ea-77ee5fa864a0-image-registry-private-configuration\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.065267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.065238 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-registry-tls\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.075675 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.075648 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mmxn\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-kube-api-access-5mmxn\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.084618 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.084587 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f61f8351-2bf3-4012-82ea-77ee5fa864a0-bound-sa-token\") pod \"image-registry-5499bf57fb-7fq4w\" (UID: \"f61f8351-2bf3-4012-82ea-77ee5fa864a0\") " pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.132745 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.132713 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.185242 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.185160 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-8swkf"] Jun 09 14:16:23.192050 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:23.192017 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeca0437_0985_4c2b_907b_08d00f3fc0bd.slice/crio-417a0fb0c36ede548a692cda91a5ae92eb7a1ce93c9def5db0bbc3bb8d4a395f WatchSource:0}: Error finding container 417a0fb0c36ede548a692cda91a5ae92eb7a1ce93c9def5db0bbc3bb8d4a395f: Status 404 returned error can't find the container with id 417a0fb0c36ede548a692cda91a5ae92eb7a1ce93c9def5db0bbc3bb8d4a395f Jun 09 14:16:23.261680 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.261651 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5499bf57fb-7fq4w"] Jun 09 14:16:23.264343 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:23.264315 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61f8351_2bf3_4012_82ea_77ee5fa864a0.slice/crio-aa378ea287800deb607ad9a1ff7ce3a10519b0bbe721ec5d5a9c2119ad5788fb WatchSource:0}: Error finding container aa378ea287800deb607ad9a1ff7ce3a10519b0bbe721ec5d5a9c2119ad5788fb: Status 404 returned error can't find the container with id aa378ea287800deb607ad9a1ff7ce3a10519b0bbe721ec5d5a9c2119ad5788fb Jun 09 14:16:23.706701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.706606 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" event={"ID":"f61f8351-2bf3-4012-82ea-77ee5fa864a0","Type":"ContainerStarted","Data":"26759d6b830f23398ef81895dc688f013fdb0f3d961eb06f83946bac4fe15ef3"} Jun 09 14:16:23.706701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.706668 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" event={"ID":"f61f8351-2bf3-4012-82ea-77ee5fa864a0","Type":"ContainerStarted","Data":"aa378ea287800deb607ad9a1ff7ce3a10519b0bbe721ec5d5a9c2119ad5788fb"} Jun 09 14:16:23.706701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.706693 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:23.707814 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.707792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-8swkf" event={"ID":"eeca0437-0985-4c2b-907b-08d00f3fc0bd","Type":"ContainerStarted","Data":"417a0fb0c36ede548a692cda91a5ae92eb7a1ce93c9def5db0bbc3bb8d4a395f"} Jun 09 14:16:23.727802 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:23.727743 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" podStartSLOduration=1.727697093 podStartE2EDuration="1.727697093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:23.726801411 +0000 UTC m=+248.373548105" watchObservedRunningTime="2026-06-09 14:16:23.727697093 +0000 UTC m=+248.374443786" Jun 09 14:16:24.171757 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.171713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:16:24.174486 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.174459 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8ebca47e-d091-429d-a238-2583e0da2bd3-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6g4j5\" (UID: \"8ebca47e-d091-429d-a238-2583e0da2bd3\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:16:24.272192 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.272156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:16:24.275197 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.275170 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef8f335f-ed20-433c-85ad-b8199bee7c60-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-52gfq\" (UID: \"ef8f335f-ed20-433c-85ad-b8199bee7c60\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:16:24.474776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.474699 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pvskr\"" Jun 09 14:16:24.483523 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.483493 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" Jun 09 14:16:24.573097 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.573067 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gcq2g\"" Jun 09 14:16:24.582075 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.582047 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" Jun 09 14:16:24.627803 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.627771 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5"] Jun 09 14:16:24.631054 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:24.631022 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ebca47e_d091_429d_a238_2583e0da2bd3.slice/crio-84dcc0885e95b3deb918ec92375426f097fcb1ebc38dcad426ea55129a26fe73 WatchSource:0}: Error finding container 84dcc0885e95b3deb918ec92375426f097fcb1ebc38dcad426ea55129a26fe73: Status 404 returned error can't find the container with id 84dcc0885e95b3deb918ec92375426f097fcb1ebc38dcad426ea55129a26fe73 Jun 09 14:16:24.712817 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.712780 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" event={"ID":"8ebca47e-d091-429d-a238-2583e0da2bd3","Type":"ContainerStarted","Data":"84dcc0885e95b3deb918ec92375426f097fcb1ebc38dcad426ea55129a26fe73"} Jun 09 14:16:24.722341 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:24.722314 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq"] Jun 09 14:16:24.725551 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:24.725490 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8f335f_ed20_433c_85ad_b8199bee7c60.slice/crio-bf7e5be2d8106860ce570b5961c52e1b78537a14091b91def875f81cb88f66f9 WatchSource:0}: Error finding container bf7e5be2d8106860ce570b5961c52e1b78537a14091b91def875f81cb88f66f9: Status 404 returned error can't find the container with id bf7e5be2d8106860ce570b5961c52e1b78537a14091b91def875f81cb88f66f9 Jun 09 14:16:25.721119 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:25.721025 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" event={"ID":"ef8f335f-ed20-433c-85ad-b8199bee7c60","Type":"ContainerStarted","Data":"bf7e5be2d8106860ce570b5961c52e1b78537a14091b91def875f81cb88f66f9"} Jun 09 14:16:26.693777 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.693736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:16:26.696290 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.696259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13916a76-8368-4bc7-b6d0-55615e3aab9e-metrics-certs\") pod \"network-metrics-daemon-hsxpw\" (UID: \"13916a76-8368-4bc7-b6d0-55615e3aab9e\") " pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:16:26.726246 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.726216 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" event={"ID":"8ebca47e-d091-429d-a238-2583e0da2bd3","Type":"ContainerStarted","Data":"7e472794c146d72668a18b796b9fc878c39172fa60b02890fa06c751841457c4"} Jun 09 14:16:26.740750 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.740604 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6g4j5" podStartSLOduration=33.367980252 podStartE2EDuration="34.740582845s" totalImagesPullingTime="1.372602593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:26.739572915 +0000 UTC m=+251.386319620" watchObservedRunningTime="2026-06-09 14:16:26.740582845 +0000 UTC m=+251.387329539" Jun 09 14:16:26.833992 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.833956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:26.875529 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.875502 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xvhx5\"" Jun 09 14:16:26.884570 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.884534 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hsxpw" Jun 09 14:16:26.933997 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:26.933916 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:27.023862 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.023497 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hsxpw"] Jun 09 14:16:27.027159 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:27.027124 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13916a76_8368_4bc7_b6d0_55615e3aab9e.slice/crio-3cbbecb27fc59858e1416728e9ee734cf29eb07d82129b6bdadd90c9576a523d WatchSource:0}: Error finding container 3cbbecb27fc59858e1416728e9ee734cf29eb07d82129b6bdadd90c9576a523d: Status 404 returned error can't find the container with id 3cbbecb27fc59858e1416728e9ee734cf29eb07d82129b6bdadd90c9576a523d Jun 09 14:16:27.059777 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.059741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:27.256619 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.256585 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:27.264031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.264005 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8"] Jun 09 14:16:27.268173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.268158 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:27.270521 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.270501 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 09 14:16:27.270521 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.270518 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lkw8n\"" Jun 09 14:16:27.275912 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.275885 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8"] Jun 09 14:16:27.400746 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.400710 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/89191b49-2d6c-45a0-a2b0-abc00566415c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-btpq8\" (UID: \"89191b49-2d6c-45a0-a2b0-abc00566415c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:27.501232 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.501192 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/89191b49-2d6c-45a0-a2b0-abc00566415c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-btpq8\" (UID: \"89191b49-2d6c-45a0-a2b0-abc00566415c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:27.501461 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:27.501394 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 09 14:16:27.501958 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:27.501506 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89191b49-2d6c-45a0-a2b0-abc00566415c-tls-certificates podName:89191b49-2d6c-45a0-a2b0-abc00566415c nodeName:}" failed. No retries permitted until 2026-06-09 14:16:28.001484017 +0000 UTC m=+252.648230703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/89191b49-2d6c-45a0-a2b0-abc00566415c-tls-certificates") pod "prometheus-operator-admission-webhook-5b6b8f594-btpq8" (UID: "89191b49-2d6c-45a0-a2b0-abc00566415c") : secret "prometheus-operator-admission-webhook-tls" not found Jun 09 14:16:27.731136 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.731103 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hsxpw" event={"ID":"13916a76-8368-4bc7-b6d0-55615e3aab9e","Type":"ContainerStarted","Data":"3cbbecb27fc59858e1416728e9ee734cf29eb07d82129b6bdadd90c9576a523d"} Jun 09 14:16:27.732816 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.732774 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" event={"ID":"ef8f335f-ed20-433c-85ad-b8199bee7c60","Type":"ContainerStarted","Data":"1eddd69d97b9ef8aff6208319a5c30ecedaa3d411529ea90988151774ed967be"} Jun 09 14:16:27.745339 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.745315 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:27.748868 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:27.748829 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-52gfq" podStartSLOduration=33.75491094 podStartE2EDuration="35.748784822s" totalImagesPullingTime="1.993873882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:27.747867523 +0000 UTC m=+252.394614217" watchObservedRunningTime="2026-06-09 14:16:27.748784822 +0000 UTC m=+252.395531514" Jun 09 14:16:28.004868 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.004764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/89191b49-2d6c-45a0-a2b0-abc00566415c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-btpq8\" (UID: \"89191b49-2d6c-45a0-a2b0-abc00566415c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:28.008254 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.008228 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/89191b49-2d6c-45a0-a2b0-abc00566415c-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-btpq8\" (UID: \"89191b49-2d6c-45a0-a2b0-abc00566415c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:28.178459 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.178425 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:28.328448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.328392 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8"] Jun 09 14:16:28.331350 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:28.331319 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89191b49_2d6c_45a0_a2b0_abc00566415c.slice/crio-626c0e72f47c0b6400dc75efa0b83daf7f333516ba3c7a13b3d9de440cb7aca1 WatchSource:0}: Error finding container 626c0e72f47c0b6400dc75efa0b83daf7f333516ba3c7a13b3d9de440cb7aca1: Status 404 returned error can't find the container with id 626c0e72f47c0b6400dc75efa0b83daf7f333516ba3c7a13b3d9de440cb7aca1 Jun 09 14:16:28.739082 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.739047 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hsxpw" event={"ID":"13916a76-8368-4bc7-b6d0-55615e3aab9e","Type":"ContainerStarted","Data":"f4d5d5e16c3724d500d217367962ac713962896a7817aa0bb90a498197d3f183"} Jun 09 14:16:28.739512 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.739116 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hsxpw" event={"ID":"13916a76-8368-4bc7-b6d0-55615e3aab9e","Type":"ContainerStarted","Data":"19ba4aa8c85f0c8a898fd93d174f99d85bdaefca68868982b55d43794061f694"} Jun 09 14:16:28.740384 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.740357 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" event={"ID":"89191b49-2d6c-45a0-a2b0-abc00566415c","Type":"ContainerStarted","Data":"626c0e72f47c0b6400dc75efa0b83daf7f333516ba3c7a13b3d9de440cb7aca1"} Jun 09 14:16:28.753963 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.753900 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hsxpw" podStartSLOduration=251.818454571 podStartE2EDuration="4m12.753881456s" totalImagesPullingTime="935.426885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:28.753552305 +0000 UTC m=+253.400299010" watchObservedRunningTime="2026-06-09 14:16:28.753881456 +0000 UTC m=+253.400628152" Jun 09 14:16:28.840618 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.840584 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:28.897166 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:28.897129 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:29.746289 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:29.746250 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" event={"ID":"89191b49-2d6c-45a0-a2b0-abc00566415c","Type":"ContainerStarted","Data":"4442c84f2af4d5d8779ec5c72b2f3f8747e9948930b053c14c46d700982f0a35"} Jun 09 14:16:29.746783 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:29.746581 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:29.752291 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:29.752267 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" Jun 09 14:16:29.761679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:29.761625 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-btpq8" podStartSLOduration=1.835518983 podStartE2EDuration="2.76160606s" totalImagesPullingTime="926.087077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:29.759834151 +0000 UTC m=+254.406580857" watchObservedRunningTime="2026-06-09 14:16:29.76160606 +0000 UTC m=+254.408352747" Jun 09 14:16:29.958756 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:29.958711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:30.015560 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:30.015452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:30.288837 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:30.288752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:30.355165 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:30.355129 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:30.706072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:30.705943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:32.719365 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:32.719320 2568 patch_prober.go:28] interesting pod/image-registry-868bcbfc76-pjd4b container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 14:16:32.719801 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:32.719385 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" podUID="44896588-ab56-415a-a983-5aec7314129a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 14:16:32.901147 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:32.901108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:34.156854 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:34.156827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:34.210551 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:34.210518 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.402998 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.402959 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.506274 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.506239 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.649190 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.649158 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.673921 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.673829 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2"] Jun 09 14:16:35.677489 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.677460 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.679953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.679927 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 09 14:16:35.679953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.679944 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4l28x\"" Jun 09 14:16:35.680144 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.680054 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 09 14:16:35.680144 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.680085 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 14:16:35.687174 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.687150 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t2w69"] Jun 09 14:16:35.690675 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.690653 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2"] Jun 09 14:16:35.690822 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.690781 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.693994 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.693974 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 14:16:35.694290 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.693972 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 14:16:35.694290 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.694025 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fjqn6\"" Jun 09 14:16:35.694290 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.694215 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp"] Jun 09 14:16:35.694785 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.694600 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 14:16:35.698725 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.698706 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.701177 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.701153 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 09 14:16:35.701485 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.701463 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 09 14:16:35.701620 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.701593 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-dtn52\"" Jun 09 14:16:35.701733 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.701712 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 09 14:16:35.708786 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.708574 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp"] Jun 09 14:16:35.737530 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.737496 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.772532 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.772499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.772532 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.772541 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.772743 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.772601 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j5tx\" (UniqueName: \"kubernetes.io/projected/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-kube-api-access-2j5tx\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.772743 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.772686 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.806998 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.806962 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.873327 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873288 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsm24\" (UniqueName: \"kubernetes.io/projected/04d51df8-e3ec-4d08-88a6-6899e78091d2-kube-api-access-zsm24\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873337 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-sys\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-wtmp\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.873536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.873536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.873536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx5nv\" (UniqueName: \"kubernetes.io/projected/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-api-access-mx5nv\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873581 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-tls\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j5tx\" (UniqueName: \"kubernetes.io/projected/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-kube-api-access-2j5tx\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873654 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-textfile\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04d51df8-e3ec-4d08-88a6-6899e78091d2-metrics-client-ca\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.873852 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-root\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.874323 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.873865 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.874323 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.874288 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.876515 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.876490 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.876654 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.876567 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.884909 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.884857 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j5tx\" (UniqueName: \"kubernetes.io/projected/a7b4fad4-a36f-4027-bb1e-8dd738bf390f-kube-api-access-2j5tx\") pod \"openshift-state-metrics-65f78d5c66-dpxx2\" (UID: \"a7b4fad4-a36f-4027-bb1e-8dd738bf390f\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:35.907673 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.907641 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:35.975083 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975083 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975083 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975348 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975105 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04d51df8-e3ec-4d08-88a6-6899e78091d2-metrics-client-ca\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975348 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975155 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-root\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975348 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975599 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975566 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975664 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975580 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-root\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975664 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsm24\" (UniqueName: \"kubernetes.io/projected/04d51df8-e3ec-4d08-88a6-6899e78091d2-kube-api-access-zsm24\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975781 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-sys\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975781 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-wtmp\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975781 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975781 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975752 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975759 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx5nv\" (UniqueName: \"kubernetes.io/projected/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-api-access-mx5nv\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975858 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-tls\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.975967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975918 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.975967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.975930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-textfile\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.976230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.976184 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-textfile\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.976286 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.976263 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-sys\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.976286 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.976272 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04d51df8-e3ec-4d08-88a6-6899e78091d2-metrics-client-ca\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.976370 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.976306 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.976445 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.976388 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-wtmp\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.978508 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.978478 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.978508 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.978499 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04d51df8-e3ec-4d08-88a6-6899e78091d2-node-exporter-tls\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.978690 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.978666 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.978824 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.978796 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.983267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.983243 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx5nv\" (UniqueName: \"kubernetes.io/projected/01920cf4-dcdc-4e0e-81be-bacc6278e1e8-kube-api-access-mx5nv\") pod \"kube-state-metrics-57bbf8bfb5-btdtp\" (UID: \"01920cf4-dcdc-4e0e-81be-bacc6278e1e8\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:35.983449 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.983428 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsm24\" (UniqueName: \"kubernetes.io/projected/04d51df8-e3ec-4d08-88a6-6899e78091d2-kube-api-access-zsm24\") pod \"node-exporter-t2w69\" (UID: \"04d51df8-e3ec-4d08-88a6-6899e78091d2\") " pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:35.989245 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:35.989225 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" Jun 09 14:16:36.003633 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.003609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t2w69" Jun 09 14:16:36.009257 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.009221 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:36.011908 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.011886 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" Jun 09 14:16:36.020191 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:36.020149 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d51df8_e3ec_4d08_88a6_6899e78091d2.slice/crio-8c475f38ca2711ca5edeb888de811d5e6f108811783b90c262de9f8465389239 WatchSource:0}: Error finding container 8c475f38ca2711ca5edeb888de811d5e6f108811783b90c262de9f8465389239: Status 404 returned error can't find the container with id 8c475f38ca2711ca5edeb888de811d5e6f108811783b90c262de9f8465389239 Jun 09 14:16:36.109314 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.109242 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:36.157116 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.157083 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2"] Jun 09 14:16:36.160463 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:36.160430 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b4fad4_a36f_4027_bb1e_8dd738bf390f.slice/crio-d8041308da3238b7390b4adcd9300ebf1595aa2a8cf776bd7115cb6d0c4fba85 WatchSource:0}: Error finding container d8041308da3238b7390b4adcd9300ebf1595aa2a8cf776bd7115cb6d0c4fba85: Status 404 returned error can't find the container with id d8041308da3238b7390b4adcd9300ebf1595aa2a8cf776bd7115cb6d0c4fba85 Jun 09 14:16:36.177974 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.177950 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp"] Jun 09 14:16:36.181261 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:36.181239 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01920cf4_dcdc_4e0e_81be_bacc6278e1e8.slice/crio-5799b69673c2cc1cb4310abdb3cbe7e4d9ac465e3bffe68c0e3d0f95b3df3f3d WatchSource:0}: Error finding container 5799b69673c2cc1cb4310abdb3cbe7e4d9ac465e3bffe68c0e3d0f95b3df3f3d: Status 404 returned error can't find the container with id 5799b69673c2cc1cb4310abdb3cbe7e4d9ac465e3bffe68c0e3d0f95b3df3f3d Jun 09 14:16:36.440925 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.440886 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:36.638575 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.638453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:36.719315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.719249 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:36.787819 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.787777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" event={"ID":"a7b4fad4-a36f-4027-bb1e-8dd738bf390f","Type":"ContainerStarted","Data":"73b09769dcd2d5d3eaa47464f9a462cc6fe7b27e58c6c36a1aa7057d0f65108f"} Jun 09 14:16:36.788001 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.787905 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" event={"ID":"a7b4fad4-a36f-4027-bb1e-8dd738bf390f","Type":"ContainerStarted","Data":"83e2fabd271a17aba2f3c8166c650bd88378e9e5a300b1cdd914fc9385f65e10"} Jun 09 14:16:36.788001 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.787954 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" event={"ID":"a7b4fad4-a36f-4027-bb1e-8dd738bf390f","Type":"ContainerStarted","Data":"d8041308da3238b7390b4adcd9300ebf1595aa2a8cf776bd7115cb6d0c4fba85"} Jun 09 14:16:36.789522 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.789491 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2w69" event={"ID":"04d51df8-e3ec-4d08-88a6-6899e78091d2","Type":"ContainerStarted","Data":"8c475f38ca2711ca5edeb888de811d5e6f108811783b90c262de9f8465389239"} Jun 09 14:16:36.790783 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.790760 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" event={"ID":"01920cf4-dcdc-4e0e-81be-bacc6278e1e8","Type":"ContainerStarted","Data":"5799b69673c2cc1cb4310abdb3cbe7e4d9ac465e3bffe68c0e3d0f95b3df3f3d"} Jun 09 14:16:36.839276 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:36.839238 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:38.571609 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.571573 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:38.638199 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.638159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:38.642595 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.642563 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5584564fcf-g6jt7"] Jun 09 14:16:38.646661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.646631 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.649435 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649308 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 09 14:16:38.649558 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649443 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 09 14:16:38.649615 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649573 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fgsnb\"" Jun 09 14:16:38.649670 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649638 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 09 14:16:38.649789 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649753 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bs5rkv099h5hg\"" Jun 09 14:16:38.649912 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649794 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 09 14:16:38.649978 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.649957 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 09 14:16:38.659059 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.659035 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5584564fcf-g6jt7"] Jun 09 14:16:38.702725 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.702690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:38.703775 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703753 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.703874 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703793 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-tls\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.703874 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.703874 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48320bc1-3572-425f-aaf0-7b389a9a1568-metrics-client-ca\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.704003 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703887 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdgjz\" (UniqueName: \"kubernetes.io/projected/48320bc1-3572-425f-aaf0-7b389a9a1568-kube-api-access-gdgjz\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.704003 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703948 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.704003 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.703987 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-grpc-tls\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.704116 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.704014 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804624 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48320bc1-3572-425f-aaf0-7b389a9a1568-metrics-client-ca\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdgjz\" (UniqueName: \"kubernetes.io/projected/48320bc1-3572-425f-aaf0-7b389a9a1568-kube-api-access-gdgjz\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804734 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-grpc-tls\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804817 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.804916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.805136 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.804921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-tls\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.806474 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.806448 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48320bc1-3572-425f-aaf0-7b389a9a1568-metrics-client-ca\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.808056 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.807996 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-tls\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.808056 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.808005 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.808307 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.808280 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.808913 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.808872 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.809066 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.809017 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.809327 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.809304 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/48320bc1-3572-425f-aaf0-7b389a9a1568-secret-grpc-tls\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.814642 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.814620 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdgjz\" (UniqueName: \"kubernetes.io/projected/48320bc1-3572-425f-aaf0-7b389a9a1568-kube-api-access-gdgjz\") pod \"thanos-querier-5584564fcf-g6jt7\" (UID: \"48320bc1-3572-425f-aaf0-7b389a9a1568\") " pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.959596 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.959490 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:38.993940 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:38.993895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:39.104513 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:39.104480 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:39.153233 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:39.153196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:40.432039 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.432006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:40.437726 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.437701 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6"] Jun 09 14:16:40.440960 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.440941 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.443345 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.443320 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 09 14:16:40.443475 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.443420 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pvq9h\"" Jun 09 14:16:40.450436 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.450377 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6"] Jun 09 14:16:40.486266 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.486227 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:40.521468 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.521424 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmhrx\" (UniqueName: \"kubernetes.io/projected/3dc5e069-b16e-44a8-bc5e-840ac232447f-kube-api-access-fmhrx\") pod \"monitoring-plugin-76b68cc874-mw5w6\" (UID: \"3dc5e069-b16e-44a8-bc5e-840ac232447f\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.521648 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.521602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dc5e069-b16e-44a8-bc5e-840ac232447f-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-mw5w6\" (UID: \"3dc5e069-b16e-44a8-bc5e-840ac232447f\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.590457 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.590425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:40.622727 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.622684 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dc5e069-b16e-44a8-bc5e-840ac232447f-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-mw5w6\" (UID: \"3dc5e069-b16e-44a8-bc5e-840ac232447f\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.622925 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.622754 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmhrx\" (UniqueName: \"kubernetes.io/projected/3dc5e069-b16e-44a8-bc5e-840ac232447f-kube-api-access-fmhrx\") pod \"monitoring-plugin-76b68cc874-mw5w6\" (UID: \"3dc5e069-b16e-44a8-bc5e-840ac232447f\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.625559 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.625531 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dc5e069-b16e-44a8-bc5e-840ac232447f-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-mw5w6\" (UID: \"3dc5e069-b16e-44a8-bc5e-840ac232447f\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.631189 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.631166 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmhrx\" (UniqueName: \"kubernetes.io/projected/3dc5e069-b16e-44a8-bc5e-840ac232447f-kube-api-access-fmhrx\") pod \"monitoring-plugin-76b68cc874-mw5w6\" (UID: \"3dc5e069-b16e-44a8-bc5e-840ac232447f\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:40.753616 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:40.753577 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:41.395070 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.395032 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76579bdfcb-qhbrr"] Jun 09 14:16:41.398316 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.398298 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.400832 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.400807 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 09 14:16:41.401734 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.401709 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 09 14:16:41.401894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.401715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h5lpd\"" Jun 09 14:16:41.402251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.402051 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 09 14:16:41.402251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.402056 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 09 14:16:41.402251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.402181 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 09 14:16:41.411448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.411422 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76579bdfcb-qhbrr"] Jun 09 14:16:41.413212 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.413186 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 09 14:16:41.430603 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-service-ca\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.430748 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-serving-cert\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.430748 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-config\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.430748 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430726 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-oauth-config\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.430911 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-trusted-ca-bundle\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.430911 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430857 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-oauth-serving-cert\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.430996 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.430908 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgplm\" (UniqueName: \"kubernetes.io/projected/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-kube-api-access-wgplm\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.531988 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.531944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-serving-cert\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.531988 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.531995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-config\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.532579 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.532031 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-oauth-config\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.532579 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.532090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-trusted-ca-bundle\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.532579 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.532219 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-oauth-serving-cert\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.532579 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.532293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgplm\" (UniqueName: \"kubernetes.io/projected/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-kube-api-access-wgplm\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.532579 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.532373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-service-ca\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.532969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.532936 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-config\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.533120 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.533083 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-trusted-ca-bundle\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.533227 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.533201 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-oauth-serving-cert\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.533425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.533374 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-service-ca\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.534960 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.534936 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-serving-cert\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.535065 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.535014 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-oauth-config\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.540659 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.540633 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgplm\" (UniqueName: \"kubernetes.io/projected/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-kube-api-access-wgplm\") pod \"console-76579bdfcb-qhbrr\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.712313 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.712231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:41.812713 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:41.812663 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:42.717969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:42.717931 2568 patch_prober.go:28] interesting pod/image-registry-868bcbfc76-pjd4b container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 14:16:42.718391 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:42.718011 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" podUID="44896588-ab56-415a-a983-5aec7314129a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 14:16:43.137572 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:43.137531 2568 patch_prober.go:28] interesting pod/image-registry-5499bf57fb-7fq4w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 14:16:43.137761 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:43.137592 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" podUID="f61f8351-2bf3-4012-82ea-77ee5fa864a0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 14:16:43.962485 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:43.962449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:44.633494 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.633322 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6"] Jun 09 14:16:44.661311 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.661278 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5584564fcf-g6jt7"] Jun 09 14:16:44.684648 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.684592 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76579bdfcb-qhbrr"] Jun 09 14:16:44.718254 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.718219 2568 patch_prober.go:28] interesting pod/image-registry-5499bf57fb-7fq4w container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 14:16:44.718490 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.718280 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" podUID="f61f8351-2bf3-4012-82ea-77ee5fa864a0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 14:16:44.826671 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.826628 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2w69" event={"ID":"04d51df8-e3ec-4d08-88a6-6899e78091d2","Type":"ContainerStarted","Data":"ee02bd307c00d6611559ed7fe92af5834af90ce88352633ef2f8a3b1cce6565c"} Jun 09 14:16:44.828762 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:44.828737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" event={"ID":"01920cf4-dcdc-4e0e-81be-bacc6278e1e8","Type":"ContainerStarted","Data":"de49bcf9003ef64174bac0201534809bab80a8baf28283e8a32e5685632b0049"} Jun 09 14:16:44.847128 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:44.847105 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a999e6_27c2_46c2_a6b9_e4a98b1bd054.slice/crio-43c3b3b7cab0e47400ff2c05761fbb06c7a0f07f8c0ce7bdc3ef4d4f54eb29a6 WatchSource:0}: Error finding container 43c3b3b7cab0e47400ff2c05761fbb06c7a0f07f8c0ce7bdc3ef4d4f54eb29a6: Status 404 returned error can't find the container with id 43c3b3b7cab0e47400ff2c05761fbb06c7a0f07f8c0ce7bdc3ef4d4f54eb29a6 Jun 09 14:16:44.847915 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:44.847866 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc5e069_b16e_44a8_bc5e_840ac232447f.slice/crio-3e07840c88fb74588de92e049f7f077604974dd8bf5082e1124bd4cad8b443e6 WatchSource:0}: Error finding container 3e07840c88fb74588de92e049f7f077604974dd8bf5082e1124bd4cad8b443e6: Status 404 returned error can't find the container with id 3e07840c88fb74588de92e049f7f077604974dd8bf5082e1124bd4cad8b443e6 Jun 09 14:16:45.166126 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.166037 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:45.266117 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.266075 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:45.843116 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.843071 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" event={"ID":"01920cf4-dcdc-4e0e-81be-bacc6278e1e8","Type":"ContainerStarted","Data":"825d33d95773ada99c341e29a6b825143b9c49f70356ebc11a9d804a5c603b8a"} Jun 09 14:16:45.843323 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.843195 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" event={"ID":"01920cf4-dcdc-4e0e-81be-bacc6278e1e8","Type":"ContainerStarted","Data":"01c3bc789e0547f2fd29bb29d27e1f27e0474a7f799479a794aa3424ee1dd76f"} Jun 09 14:16:45.852271 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.852233 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-8swkf" event={"ID":"eeca0437-0985-4c2b-907b-08d00f3fc0bd","Type":"ContainerStarted","Data":"82e59a5e9acf22623aad376334b3375e8fd0505ee78fea1153dec8fdbf13d08b"} Jun 09 14:16:45.853017 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.852720 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:45.856702 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.856669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"c9fba068c8a86c23f707203f34e64ccaed045696c9a5920e4fe0b7d552c2a124"} Jun 09 14:16:45.860906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.860833 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76579bdfcb-qhbrr" event={"ID":"10a999e6-27c2-46c2-a6b9-e4a98b1bd054","Type":"ContainerStarted","Data":"43c3b3b7cab0e47400ff2c05761fbb06c7a0f07f8c0ce7bdc3ef4d4f54eb29a6"} Jun 09 14:16:45.862506 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.862425 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" event={"ID":"3dc5e069-b16e-44a8-bc5e-840ac232447f","Type":"ContainerStarted","Data":"3e07840c88fb74588de92e049f7f077604974dd8bf5082e1124bd4cad8b443e6"} Jun 09 14:16:45.871063 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.871007 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-btdtp" podStartSLOduration=2.605410634 podStartE2EDuration="10.870986999s" totalImagesPullingTime="8.265576365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:45.869188904 +0000 UTC m=+270.515935597" watchObservedRunningTime="2026-06-09 14:16:45.870986999 +0000 UTC m=+270.517733696" Jun 09 14:16:45.876211 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.876031 2568 generic.go:356] "Generic (PLEG): container finished" podID="04d51df8-e3ec-4d08-88a6-6899e78091d2" containerID="ee02bd307c00d6611559ed7fe92af5834af90ce88352633ef2f8a3b1cce6565c" exitCode=0 Jun 09 14:16:45.882582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.882439 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-8swkf" Jun 09 14:16:45.882582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.882486 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" event={"ID":"a7b4fad4-a36f-4027-bb1e-8dd738bf390f","Type":"ContainerStarted","Data":"c3e68ecbf87076763ea965aee8a9f62ec021c74ac96423f74f3c2000aa17cb37"} Jun 09 14:16:45.882582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.882529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2w69" event={"ID":"04d51df8-e3ec-4d08-88a6-6899e78091d2","Type":"ContainerDied","Data":"ee02bd307c00d6611559ed7fe92af5834af90ce88352633ef2f8a3b1cce6565c"} Jun 09 14:16:45.894813 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.894723 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-8swkf" podStartSLOduration=2.573485035 podStartE2EDuration="23.894701286s" totalImagesPullingTime="21.321216251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:45.89243084 +0000 UTC m=+270.539177534" watchObservedRunningTime="2026-06-09 14:16:45.894701286 +0000 UTC m=+270.541447980" Jun 09 14:16:45.962425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:45.960378 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dpxx2" podStartSLOduration=2.36154664 podStartE2EDuration="10.960355812s" totalImagesPullingTime="8.598809172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:45.932228333 +0000 UTC m=+270.578975029" watchObservedRunningTime="2026-06-09 14:16:45.960355812 +0000 UTC m=+270.607102506" Jun 09 14:16:46.896582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:46.896473 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2w69" event={"ID":"04d51df8-e3ec-4d08-88a6-6899e78091d2","Type":"ContainerStarted","Data":"776c7cd62c72c2b2d7ca2e4317c944108afd1c2377932c78e161b2b47a43b8a0"} Jun 09 14:16:46.896582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:46.896529 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2w69" event={"ID":"04d51df8-e3ec-4d08-88a6-6899e78091d2","Type":"ContainerStarted","Data":"928d6924757da4333efa2ba27a697db1599154ba6f89e53990abbe77f552f34e"} Jun 09 14:16:46.921877 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:46.920823 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t2w69" podStartSLOduration=3.381130559 podStartE2EDuration="11.920789145s" totalImagesPullingTime="8.420360265s" totalInitContainerRuntime="119.298321ms" isStatefulPod=true podCreationTimestamp="2026-06-09 14:16:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:46.91929391 +0000 UTC m=+271.566040604" watchObservedRunningTime="2026-06-09 14:16:46.920789145 +0000 UTC m=+271.567535839" Jun 09 14:16:46.976910 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:46.976282 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd8f66cbf-2h9lb"] Jun 09 14:16:47.014804 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.014766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.015424 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.014663 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd8f66cbf-2h9lb"] Jun 09 14:16:47.091049 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.090991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-config\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.091049 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.091042 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-service-ca\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.091354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.091082 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-oauth-serving-cert\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.091354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.091144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-serving-cert\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.091354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.091181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57hdw\" (UniqueName: \"kubernetes.io/projected/3bccdab9-8eba-41df-8f63-374c1f032c5a-kube-api-access-57hdw\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.091354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.091217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-oauth-config\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.091354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.091287 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-trusted-ca-bundle\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-oauth-config\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192414 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-trusted-ca-bundle\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192482 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-config\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-service-ca\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192528 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-oauth-serving-cert\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-serving-cert\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.192695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.192617 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57hdw\" (UniqueName: \"kubernetes.io/projected/3bccdab9-8eba-41df-8f63-374c1f032c5a-kube-api-access-57hdw\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.193648 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.193484 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-trusted-ca-bundle\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.193648 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.193577 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-oauth-serving-cert\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.193817 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.193724 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-service-ca\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.194115 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.194092 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-config\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.195935 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.195909 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-oauth-config\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.196763 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.196723 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-serving-cert\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.208309 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.208283 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57hdw\" (UniqueName: \"kubernetes.io/projected/3bccdab9-8eba-41df-8f63-374c1f032c5a-kube-api-access-57hdw\") pod \"console-cd8f66cbf-2h9lb\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.328257 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.327493 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:47.735040 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:47.734761 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" podUID="44896588-ab56-415a-a983-5aec7314129a" containerName="registry" containerID="cri-o://f3b4828fd884db63a74d58f3ec2f6224b0f3d414b46ef8c68dd6f4187ea82229" gracePeriod=30 Jun 09 14:16:48.908253 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:48.908088 2568 generic.go:356] "Generic (PLEG): container finished" podID="44896588-ab56-415a-a983-5aec7314129a" containerID="f3b4828fd884db63a74d58f3ec2f6224b0f3d414b46ef8c68dd6f4187ea82229" exitCode=0 Jun 09 14:16:48.908253 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:48.908196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" event={"ID":"44896588-ab56-415a-a983-5aec7314129a","Type":"ContainerDied","Data":"f3b4828fd884db63a74d58f3ec2f6224b0f3d414b46ef8c68dd6f4187ea82229"} Jun 09 14:16:48.955751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:48.955719 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:49.009851 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.009759 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:49.845997 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.845964 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918562 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9vfp\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-kube-api-access-h9vfp\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918604 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-installation-pull-secrets\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918633 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-image-registry-private-configuration\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918780 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918810 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-registry-certificates\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918892 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-trusted-ca\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.918953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918920 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44896588-ab56-415a-a983-5aec7314129a-ca-trust-extracted\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.919702 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.918988 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-bound-sa-token\") pod \"44896588-ab56-415a-a983-5aec7314129a\" (UID: \"44896588-ab56-415a-a983-5aec7314129a\") " Jun 09 14:16:49.922425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.919763 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:16:49.922425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.920049 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:16:49.922425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.920973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" event={"ID":"44896588-ab56-415a-a983-5aec7314129a","Type":"ContainerDied","Data":"10466f636675757e7cf74cd9d4892d9f3540931c3f65335563f963069ac196dc"} Jun 09 14:16:49.922425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.921038 2568 scope.go:117] "RemoveContainer" containerID="f3b4828fd884db63a74d58f3ec2f6224b0f3d414b46ef8c68dd6f4187ea82229" Jun 09 14:16:49.922425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.921205 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868bcbfc76-pjd4b" Jun 09 14:16:49.925734 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.925629 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:16:49.926439 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.926305 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-kube-api-access-h9vfp" (OuterVolumeSpecName: "kube-api-access-h9vfp") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "kube-api-access-h9vfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:16:49.928391 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.928344 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:16:49.928509 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.928421 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:16:49.934202 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.933994 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:16:49.934788 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:49.934633 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44896588-ab56-415a-a983-5aec7314129a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "44896588-ab56-415a-a983-5aec7314129a" (UID: "44896588-ab56-415a-a983-5aec7314129a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020137 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-bound-sa-token\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020178 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9vfp\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-kube-api-access-h9vfp\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020195 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-installation-pull-secrets\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020210 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44896588-ab56-415a-a983-5aec7314129a-image-registry-private-configuration\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020224 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44896588-ab56-415a-a983-5aec7314129a-registry-tls\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020238 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-registry-certificates\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020250 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44896588-ab56-415a-a983-5aec7314129a-trusted-ca\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.020293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.020263 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44896588-ab56-415a-a983-5aec7314129a-ca-trust-extracted\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:16:50.040718 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.040533 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd8f66cbf-2h9lb"] Jun 09 14:16:50.165327 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:16:50.165251 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bccdab9_8eba_41df_8f63_374c1f032c5a.slice/crio-6315c536276c55b4984262033a62a071cab6d8de8a9feb3cf2ed0a043c9f8551 WatchSource:0}: Error finding container 6315c536276c55b4984262033a62a071cab6d8de8a9feb3cf2ed0a043c9f8551: Status 404 returned error can't find the container with id 6315c536276c55b4984262033a62a071cab6d8de8a9feb3cf2ed0a043c9f8551 Jun 09 14:16:50.265987 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.265956 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-868bcbfc76-pjd4b"] Jun 09 14:16:50.270373 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.270176 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-868bcbfc76-pjd4b"] Jun 09 14:16:50.278620 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.278591 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:50.379812 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.379781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:50.931875 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.931835 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"673b06cdcab78c90532160e3a78eb3fe36f3e8afe9db8c3a6d0b5c24930e72d4"} Jun 09 14:16:50.932430 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.931919 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"0d4681ebbd77f0b55cf22701ccee57f784b8d1261168be801920d0a154301169"} Jun 09 14:16:50.932430 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.931957 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"75a03ff0c98a4dbbf4ba9df5ab7a5e8483966557dd265bbfb2123e1f4de2d44c"} Jun 09 14:16:50.934677 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.934649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76579bdfcb-qhbrr" event={"ID":"10a999e6-27c2-46c2-a6b9-e4a98b1bd054","Type":"ContainerStarted","Data":"621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc"} Jun 09 14:16:50.938455 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.938380 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd8f66cbf-2h9lb" event={"ID":"3bccdab9-8eba-41df-8f63-374c1f032c5a","Type":"ContainerStarted","Data":"6d7680d220afff807a3a466462a82b7fdb282b81e93458ccdb5a843bf3214dee"} Jun 09 14:16:50.938648 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.938473 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd8f66cbf-2h9lb" event={"ID":"3bccdab9-8eba-41df-8f63-374c1f032c5a","Type":"ContainerStarted","Data":"6315c536276c55b4984262033a62a071cab6d8de8a9feb3cf2ed0a043c9f8551"} Jun 09 14:16:50.942679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.940581 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" event={"ID":"3dc5e069-b16e-44a8-bc5e-840ac232447f","Type":"ContainerStarted","Data":"e75f5cac31e6d6ed62bf3e417202c5a51197d3c395d6172bdfc37f8c4c7df48c"} Jun 09 14:16:50.942679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.941565 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:50.947763 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.947734 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" Jun 09 14:16:50.953848 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.953791 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76579bdfcb-qhbrr" podStartSLOduration=4.620630583 podStartE2EDuration="9.953772306s" totalImagesPullingTime="5.333141723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:50.951104943 +0000 UTC m=+275.597851637" watchObservedRunningTime="2026-06-09 14:16:50.953772306 +0000 UTC m=+275.600518999" Jun 09 14:16:50.971642 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.970932 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd8f66cbf-2h9lb" podStartSLOduration=4.970910336 podStartE2EDuration="4.970910336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:50.969578471 +0000 UTC m=+275.616325164" watchObservedRunningTime="2026-06-09 14:16:50.970910336 +0000 UTC m=+275.617657030" Jun 09 14:16:50.984166 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:50.984111 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-mw5w6" podStartSLOduration=5.97575386 podStartE2EDuration="10.984093158s" totalImagesPullingTime="5.008339298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:50.983700425 +0000 UTC m=+275.630447119" watchObservedRunningTime="2026-06-09 14:16:50.984093158 +0000 UTC m=+275.630839851" Jun 09 14:16:51.712875 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.712844 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:51.713082 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.712933 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:16:51.714461 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.714437 2568 patch_prober.go:28] interesting pod/console-76579bdfcb-qhbrr container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" start-of-body= Jun 09 14:16:51.714582 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.714478 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76579bdfcb-qhbrr" podUID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" containerName="console" probeResult="failure" output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" Jun 09 14:16:51.877500 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.877470 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44896588-ab56-415a-a983-5aec7314129a" path="/var/lib/kubelet/pods/44896588-ab56-415a-a983-5aec7314129a/volumes" Jun 09 14:16:51.949116 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.949080 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"478969da884ce4c71b3a81c6d602fb67495434423836119a63dd8cd1e967bc33"} Jun 09 14:16:51.949494 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:51.949168 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"005c2be10a4e33c86b94a9ffef954d584450968c73fccdb904130a0495151053"} Jun 09 14:16:52.958129 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:52.958093 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" event={"ID":"48320bc1-3572-425f-aaf0-7b389a9a1568","Type":"ContainerStarted","Data":"b2b6dc8b2022fba208245019d7eb7fdd2effa784ed3350d09f64e16fd0936d16"} Jun 09 14:16:52.980877 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:52.980752 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" podStartSLOduration=8.729603997 podStartE2EDuration="14.98072776s" totalImagesPullingTime="6.251123763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:16:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:16:52.979722929 +0000 UTC m=+277.626469624" watchObservedRunningTime="2026-06-09 14:16:52.98072776 +0000 UTC m=+277.627474454" Jun 09 14:16:53.138087 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:53.138037 2568 patch_prober.go:28] interesting pod/image-registry-5499bf57fb-7fq4w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 14:16:53.138256 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:53.138099 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" podUID="f61f8351-2bf3-4012-82ea-77ee5fa864a0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 14:16:53.962315 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:53.962284 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:16:54.717929 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:54.717897 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5499bf57fb-7fq4w" Jun 09 14:16:54.833585 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:54.833547 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:56.361630 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:56.361597 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:57.328158 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:57.328129 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:57.328352 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:57.328179 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:57.332827 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:57.332804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:57.444425 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:16:57.444368 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-54mcq" podUID="a06ae45d-d42d-4ed3-9770-95289acac343" Jun 09 14:16:57.977458 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:57.977431 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54mcq" Jun 09 14:16:57.981248 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:57.981229 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:16:58.024700 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:58.024670 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76579bdfcb-qhbrr"] Jun 09 14:16:58.123830 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:58.123786 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:58.171608 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:58.171581 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:58.225460 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:58.225424 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:58.283177 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:58.283099 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:58.646245 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:58.646167 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:16:59.972312 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:16:59.972287 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5584564fcf-g6jt7" Jun 09 14:17:00.618954 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:00.618915 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:17:00.621366 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:00.621339 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a06ae45d-d42d-4ed3-9770-95289acac343-metrics-tls\") pod \"dns-default-54mcq\" (UID: \"a06ae45d-d42d-4ed3-9770-95289acac343\") " pod="openshift-dns/dns-default-54mcq" Jun 09 14:17:00.680608 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:00.680577 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-55pg5\"" Jun 09 14:17:00.689533 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:00.689515 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-54mcq" Jun 09 14:17:00.871360 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:00.871328 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-54mcq"] Jun 09 14:17:00.873750 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:17:00.873718 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06ae45d_d42d_4ed3_9770_95289acac343.slice/crio-d2c19bbdba1299a749519629ce281bdc3bb38097204f8d4ef84c981b4870e7ff WatchSource:0}: Error finding container d2c19bbdba1299a749519629ce281bdc3bb38097204f8d4ef84c981b4870e7ff: Status 404 returned error can't find the container with id d2c19bbdba1299a749519629ce281bdc3bb38097204f8d4ef84c981b4870e7ff Jun 09 14:17:00.988933 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:00.988903 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54mcq" event={"ID":"a06ae45d-d42d-4ed3-9770-95289acac343","Type":"ContainerStarted","Data":"d2c19bbdba1299a749519629ce281bdc3bb38097204f8d4ef84c981b4870e7ff"} Jun 09 14:17:02.157269 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:02.157238 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:02.218962 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:02.218924 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:04.002379 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:04.002295 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54mcq" event={"ID":"a06ae45d-d42d-4ed3-9770-95289acac343","Type":"ContainerStarted","Data":"76bdc6810f61aff3e351e3b9f69e19495eeef1e4da9b2439996cc53fb6e9a8ae"} Jun 09 14:17:04.002379 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:04.002371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-54mcq" event={"ID":"a06ae45d-d42d-4ed3-9770-95289acac343","Type":"ContainerStarted","Data":"51de234059e1343bc90ef751623f967f9b9b8c115043164ed4946069fb0e37f1"} Jun 09 14:17:04.002862 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:04.002459 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-54mcq" Jun 09 14:17:04.022844 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:04.022788 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-54mcq" podStartSLOduration=251.412887937 podStartE2EDuration="4m14.022775118s" totalImagesPullingTime="2.609887181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:17:04.019574492 +0000 UTC m=+288.666321186" watchObservedRunningTime="2026-06-09 14:17:04.022775118 +0000 UTC m=+288.669521832" Jun 09 14:17:04.948384 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:04.948334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:05.002749 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:05.002712 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:05.532292 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:05.532250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:11.936146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:11.936114 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:13.358237 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:13.358204 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:13.408700 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:13.408664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:14.008206 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:14.008178 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-54mcq" Jun 09 14:17:15.811508 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:15.811484 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:17:15.811918 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:15.811849 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:17:15.821440 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:15.821395 2568 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 14:17:20.268703 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:20.268668 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:20.368935 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:20.368898 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:20.519745 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:20.519662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:20.621475 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:20.621447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:20.678891 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:20.678850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:21.073765 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:21.073729 2568 generic.go:356] "Generic (PLEG): container finished" podID="023aaede-6273-4afe-8d49-115a2585690b" containerID="0af5925b9a2f4b366c852516fc2a7719204774934f95beb7d5fe81ee00a8feae" exitCode=0 Jun 09 14:17:21.073919 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:21.073798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hf2bv" event={"ID":"023aaede-6273-4afe-8d49-115a2585690b","Type":"ContainerDied","Data":"0af5925b9a2f4b366c852516fc2a7719204774934f95beb7d5fe81ee00a8feae"} Jun 09 14:17:21.074235 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:21.074214 2568 scope.go:117] "RemoveContainer" containerID="0af5925b9a2f4b366c852516fc2a7719204774934f95beb7d5fe81ee00a8feae" Jun 09 14:17:21.075287 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:21.075269 2568 generic.go:356] "Generic (PLEG): container finished" podID="55cc31f5-c7f1-440b-92e1-ef3068e777e7" containerID="97166b126959d06c08fa5ac1ca33e18de90daaaf635dcbf4ca23dc17a25e361d" exitCode=0 Jun 09 14:17:21.075368 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:21.075342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" event={"ID":"55cc31f5-c7f1-440b-92e1-ef3068e777e7","Type":"ContainerDied","Data":"97166b126959d06c08fa5ac1ca33e18de90daaaf635dcbf4ca23dc17a25e361d"} Jun 09 14:17:21.075626 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:21.075610 2568 scope.go:117] "RemoveContainer" containerID="97166b126959d06c08fa5ac1ca33e18de90daaaf635dcbf4ca23dc17a25e361d" Jun 09 14:17:22.081236 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:22.081196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hf2bv" event={"ID":"023aaede-6273-4afe-8d49-115a2585690b","Type":"ContainerStarted","Data":"be6faaaad4dd535dd52cf318d10c0397349c516767e44cf0436a25c76a81eaeb"} Jun 09 14:17:22.082911 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:22.082887 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-4m8dl" event={"ID":"55cc31f5-c7f1-440b-92e1-ef3068e777e7","Type":"ContainerStarted","Data":"7e7c04922ac8b019ecd3db66ce7a0c69283a68aedb9a91885abb1bba64bec97f"} Jun 09 14:17:22.444031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:22.443952 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:22.543372 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:22.543328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:23.047648 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.047611 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76579bdfcb-qhbrr" podUID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" containerName="console" containerID="cri-o://621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc" gracePeriod=15 Jun 09 14:17:23.129821 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.129781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:23.199376 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.199347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:23.315958 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.315888 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:17:23.411522 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411493 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-config\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.411522 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411526 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-oauth-config\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.411750 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411549 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-service-ca\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.411750 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411567 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgplm\" (UniqueName: \"kubernetes.io/projected/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-kube-api-access-wgplm\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.411750 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411721 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-serving-cert\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.411915 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411770 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-trusted-ca-bundle\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.411915 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411827 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-oauth-serving-cert\") pod \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\" (UID: \"10a999e6-27c2-46c2-a6b9-e4a98b1bd054\") " Jun 09 14:17:23.412068 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.411977 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-service-ca" (OuterVolumeSpecName: "service-ca") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:17:23.412151 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.412067 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-config" (OuterVolumeSpecName: "console-config") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:17:23.412151 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.412089 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-service-ca\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:23.412260 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.412169 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:17:23.412493 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.412468 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:17:23.413927 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.413908 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:17:23.414441 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.414382 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:17:23.414441 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.414395 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-kube-api-access-wgplm" (OuterVolumeSpecName: "kube-api-access-wgplm") pod "10a999e6-27c2-46c2-a6b9-e4a98b1bd054" (UID: "10a999e6-27c2-46c2-a6b9-e4a98b1bd054"). InnerVolumeSpecName "kube-api-access-wgplm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:17:23.512667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.512641 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-serving-cert\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:23.512667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.512662 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-trusted-ca-bundle\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:23.512667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.512671 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-oauth-serving-cert\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:23.512865 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.512680 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-config\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:23.512865 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.512690 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-console-oauth-config\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:23.512865 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:23.512700 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgplm\" (UniqueName: \"kubernetes.io/projected/10a999e6-27c2-46c2-a6b9-e4a98b1bd054-kube-api-access-wgplm\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:17:24.091915 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.091880 2568 generic.go:356] "Generic (PLEG): container finished" podID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" containerID="621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc" exitCode=0 Jun 09 14:17:24.092080 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.091921 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76579bdfcb-qhbrr" event={"ID":"10a999e6-27c2-46c2-a6b9-e4a98b1bd054","Type":"ContainerDied","Data":"621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc"} Jun 09 14:17:24.092080 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.091963 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76579bdfcb-qhbrr" event={"ID":"10a999e6-27c2-46c2-a6b9-e4a98b1bd054","Type":"ContainerDied","Data":"43c3b3b7cab0e47400ff2c05761fbb06c7a0f07f8c0ce7bdc3ef4d4f54eb29a6"} Jun 09 14:17:24.092080 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.091982 2568 scope.go:117] "RemoveContainer" containerID="621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc" Jun 09 14:17:24.092080 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.091923 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76579bdfcb-qhbrr" Jun 09 14:17:24.108468 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.108446 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76579bdfcb-qhbrr"] Jun 09 14:17:24.110371 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.110349 2568 scope.go:117] "RemoveContainer" containerID="621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc" Jun 09 14:17:24.110782 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:17:24.110753 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc\": container with ID starting with 621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc not found: ID does not exist" containerID="621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc" Jun 09 14:17:24.110861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.110790 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc"} err="failed to get container status \"621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc\": rpc error: code = NotFound desc = could not find container \"621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc\": container with ID starting with 621d95258c65ff86b61403abae965d5353978d241378b8fe4b2521013fbcfccc not found: ID does not exist" Jun 09 14:17:24.111781 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:24.111764 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76579bdfcb-qhbrr"] Jun 09 14:17:25.876312 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:25.876276 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" path="/var/lib/kubelet/pods/10a999e6-27c2-46c2-a6b9-e4a98b1bd054/volumes" Jun 09 14:17:35.271837 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:35.271741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:35.323351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:35.323255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:51.728338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:51.728292 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:51.786528 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:51.786497 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:59.800209 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:59.800122 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:17:59.855499 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:17:59.855471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:04.504415 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:04.504365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:04.553713 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:04.553682 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:08.256192 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:08.256150 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:08.309426 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:08.309370 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:08.359251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:08.359216 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:10.208288 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:10.208258 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd8f66cbf-2h9lb"] Jun 09 14:18:10.301630 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:10.301601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:10.347354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:10.347330 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:10.411710 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:10.411691 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:10.512064 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:10.512042 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:20.277090 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:20.277045 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:20.378689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:20.378651 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:35.228626 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.228587 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-cd8f66cbf-2h9lb" podUID="3bccdab9-8eba-41df-8f63-374c1f032c5a" containerName="console" containerID="cri-o://6d7680d220afff807a3a466462a82b7fdb282b81e93458ccdb5a843bf3214dee" gracePeriod=15 Jun 09 14:18:35.374640 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.374610 2568 generic.go:356] "Generic (PLEG): container finished" podID="3bccdab9-8eba-41df-8f63-374c1f032c5a" containerID="6d7680d220afff807a3a466462a82b7fdb282b81e93458ccdb5a843bf3214dee" exitCode=0 Jun 09 14:18:35.374787 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.374679 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd8f66cbf-2h9lb" event={"ID":"3bccdab9-8eba-41df-8f63-374c1f032c5a","Type":"ContainerDied","Data":"6d7680d220afff807a3a466462a82b7fdb282b81e93458ccdb5a843bf3214dee"} Jun 09 14:18:35.478360 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.478334 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:18:35.576631 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576550 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-trusted-ca-bundle\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.576631 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576590 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-serving-cert\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.576631 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576616 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-oauth-config\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.576905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576671 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-oauth-serving-cert\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.576905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576689 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57hdw\" (UniqueName: \"kubernetes.io/projected/3bccdab9-8eba-41df-8f63-374c1f032c5a-kube-api-access-57hdw\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.576905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576712 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-service-ca\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.576905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.576757 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-config\") pod \"3bccdab9-8eba-41df-8f63-374c1f032c5a\" (UID: \"3bccdab9-8eba-41df-8f63-374c1f032c5a\") " Jun 09 14:18:35.577111 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.577073 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:18:35.577274 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.577247 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:18:35.577546 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.577282 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-config" (OuterVolumeSpecName: "console-config") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:18:35.577546 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.577343 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-service-ca" (OuterVolumeSpecName: "service-ca") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 14:18:35.578979 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.578952 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:18:35.579314 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.579296 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:18:35.579373 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.579317 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bccdab9-8eba-41df-8f63-374c1f032c5a-kube-api-access-57hdw" (OuterVolumeSpecName: "kube-api-access-57hdw") pod "3bccdab9-8eba-41df-8f63-374c1f032c5a" (UID: "3bccdab9-8eba-41df-8f63-374c1f032c5a"). InnerVolumeSpecName "kube-api-access-57hdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:18:35.677610 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677573 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-oauth-serving-cert\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:35.677610 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677604 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57hdw\" (UniqueName: \"kubernetes.io/projected/3bccdab9-8eba-41df-8f63-374c1f032c5a-kube-api-access-57hdw\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:35.677610 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677614 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-service-ca\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:35.677836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677623 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-config\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:35.677836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677631 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bccdab9-8eba-41df-8f63-374c1f032c5a-trusted-ca-bundle\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:35.677836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677640 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-serving-cert\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:35.677836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:35.677648 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bccdab9-8eba-41df-8f63-374c1f032c5a-console-oauth-config\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:18:36.379703 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:36.379673 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd8f66cbf-2h9lb" event={"ID":"3bccdab9-8eba-41df-8f63-374c1f032c5a","Type":"ContainerDied","Data":"6315c536276c55b4984262033a62a071cab6d8de8a9feb3cf2ed0a043c9f8551"} Jun 09 14:18:36.380190 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:36.379706 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd8f66cbf-2h9lb" Jun 09 14:18:36.380190 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:36.379724 2568 scope.go:117] "RemoveContainer" containerID="6d7680d220afff807a3a466462a82b7fdb282b81e93458ccdb5a843bf3214dee" Jun 09 14:18:36.398873 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:36.398848 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd8f66cbf-2h9lb"] Jun 09 14:18:36.407314 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:36.407292 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd8f66cbf-2h9lb"] Jun 09 14:18:37.876107 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:37.876073 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bccdab9-8eba-41df-8f63-374c1f032c5a" path="/var/lib/kubelet/pods/3bccdab9-8eba-41df-8f63-374c1f032c5a/volumes" Jun 09 14:18:52.038672 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:52.038617 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:18:52.090169 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:18:52.090141 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:10.945095 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:10.945047 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:11.040942 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:11.040910 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:37.242438 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:37.242328 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:38.606318 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:38.606280 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:40.853666 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:40.853618 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:40.954093 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:40.954054 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:42.776936 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:42.776899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:42.880786 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:42.880755 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:42.984909 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:42.984879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:45.649495 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649457 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-cpddl"] Jun 09 14:19:45.649967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649886 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" containerName="console" Jun 09 14:19:45.649967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649904 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" containerName="console" Jun 09 14:19:45.649967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649917 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bccdab9-8eba-41df-8f63-374c1f032c5a" containerName="console" Jun 09 14:19:45.649967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649924 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bccdab9-8eba-41df-8f63-374c1f032c5a" containerName="console" Jun 09 14:19:45.649967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649942 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44896588-ab56-415a-a983-5aec7314129a" containerName="registry" Jun 09 14:19:45.649967 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.649951 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="44896588-ab56-415a-a983-5aec7314129a" containerName="registry" Jun 09 14:19:45.650254 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.650047 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a999e6-27c2-46c2-a6b9-e4a98b1bd054" containerName="console" Jun 09 14:19:45.650254 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.650058 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bccdab9-8eba-41df-8f63-374c1f032c5a" containerName="console" Jun 09 14:19:45.650254 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.650070 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="44896588-ab56-415a-a983-5aec7314129a" containerName="registry" Jun 09 14:19:45.652062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.652041 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.654447 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.654427 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 09 14:19:45.655288 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.655273 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xwd5j\"" Jun 09 14:19:45.655378 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.655288 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 09 14:19:45.662948 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.662922 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-cpddl"] Jun 09 14:19:45.720701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.720666 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvzp\" (UniqueName: \"kubernetes.io/projected/b73a86e3-d872-426c-8464-a699ad5891cb-kube-api-access-tvvzp\") pod \"cert-manager-webhook-7dc76cbb94-cpddl\" (UID: \"b73a86e3-d872-426c-8464-a699ad5891cb\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.720848 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.720760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b73a86e3-d872-426c-8464-a699ad5891cb-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-cpddl\" (UID: \"b73a86e3-d872-426c-8464-a699ad5891cb\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.821127 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.821093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b73a86e3-d872-426c-8464-a699ad5891cb-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-cpddl\" (UID: \"b73a86e3-d872-426c-8464-a699ad5891cb\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.821263 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.821155 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvzp\" (UniqueName: \"kubernetes.io/projected/b73a86e3-d872-426c-8464-a699ad5891cb-kube-api-access-tvvzp\") pod \"cert-manager-webhook-7dc76cbb94-cpddl\" (UID: \"b73a86e3-d872-426c-8464-a699ad5891cb\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.830037 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.830009 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b73a86e3-d872-426c-8464-a699ad5891cb-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-cpddl\" (UID: \"b73a86e3-d872-426c-8464-a699ad5891cb\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.830554 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.830534 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvzp\" (UniqueName: \"kubernetes.io/projected/b73a86e3-d872-426c-8464-a699ad5891cb-kube-api-access-tvvzp\") pod \"cert-manager-webhook-7dc76cbb94-cpddl\" (UID: \"b73a86e3-d872-426c-8464-a699ad5891cb\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:45.972428 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:45.972338 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:46.058696 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:46.058663 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:46.101350 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:46.101321 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-cpddl"] Jun 09 14:19:46.105251 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:19:46.105220 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb73a86e3_d872_426c_8464_a699ad5891cb.slice/crio-3f7f11d9861b8400f6ff28faa59eebb0f64ca7e66cc5a7ba5d1fbf17e3d5d374 WatchSource:0}: Error finding container 3f7f11d9861b8400f6ff28faa59eebb0f64ca7e66cc5a7ba5d1fbf17e3d5d374: Status 404 returned error can't find the container with id 3f7f11d9861b8400f6ff28faa59eebb0f64ca7e66cc5a7ba5d1fbf17e3d5d374 Jun 09 14:19:46.107604 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:46.107585 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 14:19:46.645351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:46.645299 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:46.651935 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:46.651907 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" event={"ID":"b73a86e3-d872-426c-8464-a699ad5891cb","Type":"ContainerStarted","Data":"3f7f11d9861b8400f6ff28faa59eebb0f64ca7e66cc5a7ba5d1fbf17e3d5d374"} Jun 09 14:19:47.855458 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:47.850969 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:47.910111 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:47.910070 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:49.669554 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:49.669509 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" event={"ID":"b73a86e3-d872-426c-8464-a699ad5891cb","Type":"ContainerStarted","Data":"c859fca44da4978b57c6c8b1f857a98d409accf3983acc402b54a3f128e33672"} Jun 09 14:19:49.669953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:49.669679 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:49.685700 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:49.685645 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" podStartSLOduration=1.323965058 podStartE2EDuration="4.685627839s" totalImagesPullingTime="3.361662781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:19:49.685193263 +0000 UTC m=+454.331939955" watchObservedRunningTime="2026-06-09 14:19:49.685627839 +0000 UTC m=+454.332374534" Jun 09 14:19:50.841563 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:50.841527 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:50.963683 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:50.963651 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:54.446549 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:54.446509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:55.583962 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:55.583929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:19:55.675064 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:55.675036 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-cpddl" Jun 09 14:19:55.680563 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:19:55.680536 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:02.330577 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:02.330545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:02.434584 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:02.434554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:04.487096 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:04.487063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:20.082069 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:20.082038 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:20.182765 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:20.182725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:20.694117 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:20.694082 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:20.910160 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:20.910123 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.020047 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.020020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.024746 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.024722 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw"] Jun 09 14:20:21.029366 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.029347 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.031689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.031667 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 14:20:21.031815 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.031786 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 14:20:21.031977 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.031953 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 14:20:21.032119 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.032100 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rsghx\"" Jun 09 14:20:21.032198 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.031831 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 14:20:21.048031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.047972 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw"] Jun 09 14:20:21.095455 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.095418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9lvm\" (UniqueName: \"kubernetes.io/projected/3fb57b60-574a-4454-8cf3-d6505fc762a3-kube-api-access-z9lvm\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.095833 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.095524 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3fb57b60-574a-4454-8cf3-d6505fc762a3-manifests\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.095833 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.095560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fb57b60-574a-4454-8cf3-d6505fc762a3-webhook-cert\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.095833 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.095586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fb57b60-574a-4454-8cf3-d6505fc762a3-tmp\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.095833 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.095604 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3fb57b60-574a-4454-8cf3-d6505fc762a3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.140761 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.140729 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.196267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.196239 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3fb57b60-574a-4454-8cf3-d6505fc762a3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.196431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.196270 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9lvm\" (UniqueName: \"kubernetes.io/projected/3fb57b60-574a-4454-8cf3-d6505fc762a3-kube-api-access-z9lvm\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.196431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.196323 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3fb57b60-574a-4454-8cf3-d6505fc762a3-manifests\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.196431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.196342 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fb57b60-574a-4454-8cf3-d6505fc762a3-webhook-cert\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.196431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.196362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fb57b60-574a-4454-8cf3-d6505fc762a3-tmp\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.196795 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.196768 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3fb57b60-574a-4454-8cf3-d6505fc762a3-manifests\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.198798 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.198778 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3fb57b60-574a-4454-8cf3-d6505fc762a3-tmp\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.198947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.198928 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3fb57b60-574a-4454-8cf3-d6505fc762a3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.199081 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.199063 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3fb57b60-574a-4454-8cf3-d6505fc762a3-webhook-cert\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.203997 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.203978 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9lvm\" (UniqueName: \"kubernetes.io/projected/3fb57b60-574a-4454-8cf3-d6505fc762a3-kube-api-access-z9lvm\") pod \"opendatahub-operator-controller-manager-c57c599f5-26gnw\" (UID: \"3fb57b60-574a-4454-8cf3-d6505fc762a3\") " pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.243759 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.243736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.340144 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.340073 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:21.346984 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.346955 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.453100 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.453069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.472459 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.472432 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw"] Jun 09 14:20:21.475918 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:21.475888 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb57b60_574a_4454_8cf3_d6505fc762a3.slice/crio-9d52389341a0da9cc08f9667f36b368837bc222716ddf289c836ed33031b0f58 WatchSource:0}: Error finding container 9d52389341a0da9cc08f9667f36b368837bc222716ddf289c836ed33031b0f58: Status 404 returned error can't find the container with id 9d52389341a0da9cc08f9667f36b368837bc222716ddf289c836ed33031b0f58 Jun 09 14:20:21.542043 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.542003 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.643391 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.643333 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:21.797485 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:21.797443 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" event={"ID":"3fb57b60-574a-4454-8cf3-d6505fc762a3","Type":"ContainerStarted","Data":"9d52389341a0da9cc08f9667f36b368837bc222716ddf289c836ed33031b0f58"} Jun 09 14:20:24.771019 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:24.770988 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:24.883846 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:24.883821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:25.818256 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:25.818224 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" event={"ID":"3fb57b60-574a-4454-8cf3-d6505fc762a3","Type":"ContainerStarted","Data":"8d3b57d1f15ee4db8a484a99df1ddd8207d70ca3a6eb1fe9acbf2e47d9e6c157"} Jun 09 14:20:26.824133 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:26.824096 2568 generic.go:356] "Generic (PLEG): container finished" podID="3fb57b60-574a-4454-8cf3-d6505fc762a3" containerID="8d3b57d1f15ee4db8a484a99df1ddd8207d70ca3a6eb1fe9acbf2e47d9e6c157" exitCode=0 Jun 09 14:20:26.824601 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:26.824184 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" event={"ID":"3fb57b60-574a-4454-8cf3-d6505fc762a3","Type":"ContainerDied","Data":"8d3b57d1f15ee4db8a484a99df1ddd8207d70ca3a6eb1fe9acbf2e47d9e6c157"} Jun 09 14:20:27.830219 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:27.830181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" event={"ID":"3fb57b60-574a-4454-8cf3-d6505fc762a3","Type":"ContainerStarted","Data":"62d4488df95b9d13ad61030cbbd6b0836993b0c2c149de9e1b72e104e4f0f30b"} Jun 09 14:20:27.830707 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:27.830275 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:27.850807 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:27.850733 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" podStartSLOduration=3.129645096 podStartE2EDuration="7.850717545s" totalImagesPullingTime="3.728547239s" totalInitContainerRuntime="992.52521ms" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:20:27.849366054 +0000 UTC m=+492.496112748" watchObservedRunningTime="2026-06-09 14:20:27.850717545 +0000 UTC m=+492.497464237" Jun 09 14:20:38.838024 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:38.837995 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c57c599f5-26gnw" Jun 09 14:20:40.450087 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.450052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:40.562009 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.561971 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:40.742886 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.742860 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:40.880031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.879997 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:40.909896 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.909862 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z"] Jun 09 14:20:40.920176 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.920142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:40.922600 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.922564 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 09 14:20:40.922600 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.922588 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 14:20:40.922796 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.922589 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 14:20:40.923456 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.923442 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 09 14:20:40.923456 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.923449 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xnjlx\"" Jun 09 14:20:40.929752 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.929729 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z"] Jun 09 14:20:40.957442 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.957384 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tls-certs\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:40.957619 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.957553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tmp\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:40.957660 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:40.957624 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bslfx\" (UniqueName: \"kubernetes.io/projected/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-kube-api-access-bslfx\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.058686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.058601 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bslfx\" (UniqueName: \"kubernetes.io/projected/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-kube-api-access-bslfx\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.058686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.058653 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tls-certs\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.058876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.058698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tmp\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.058876 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:41.058814 2568 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 09 14:20:41.058944 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:41.058895 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tls-certs podName:fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c nodeName:}" failed. No retries permitted until 2026-06-09 14:20:41.558871787 +0000 UTC m=+506.205618459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tls-certs") pod "kube-auth-proxy-6df8999ffb-wff2z" (UID: "fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c") : secret "kube-auth-proxy-tls" not found Jun 09 14:20:41.061081 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.061063 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tmp\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.077117 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.077087 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bslfx\" (UniqueName: \"kubernetes.io/projected/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-kube-api-access-bslfx\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.213813 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.213775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:41.390101 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.390024 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-l8fbt"] Jun 09 14:20:41.392645 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.392630 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:41.394845 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.394824 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-g52gb\"" Jun 09 14:20:41.394845 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.394836 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 09 14:20:41.402277 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.402255 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-l8fbt"] Jun 09 14:20:41.422251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.422223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:41.460361 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.460334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31edc12e-1f61-48f2-b550-628cd0e6b021-cert\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:41.460713 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.460370 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j45c\" (UniqueName: \"kubernetes.io/projected/31edc12e-1f61-48f2-b550-628cd0e6b021-kube-api-access-4j45c\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:41.517862 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.517834 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:41.561763 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.561736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tls-certs\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.561882 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.561783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31edc12e-1f61-48f2-b550-628cd0e6b021-cert\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:41.561882 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.561802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j45c\" (UniqueName: \"kubernetes.io/projected/31edc12e-1f61-48f2-b550-628cd0e6b021-kube-api-access-4j45c\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:41.561960 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:41.561887 2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 09 14:20:41.561960 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:41.561943 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31edc12e-1f61-48f2-b550-628cd0e6b021-cert podName:31edc12e-1f61-48f2-b550-628cd0e6b021 nodeName:}" failed. No retries permitted until 2026-06-09 14:20:42.061928743 +0000 UTC m=+506.708675414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/31edc12e-1f61-48f2-b550-628cd0e6b021-cert") pod "odh-model-controller-6dc94f689f-l8fbt" (UID: "31edc12e-1f61-48f2-b550-628cd0e6b021") : secret "odh-model-controller-webhook-cert" not found Jun 09 14:20:41.564291 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.564266 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c-tls-certs\") pod \"kube-auth-proxy-6df8999ffb-wff2z\" (UID: \"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c\") " pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.570121 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.570098 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j45c\" (UniqueName: \"kubernetes.io/projected/31edc12e-1f61-48f2-b550-628cd0e6b021-kube-api-access-4j45c\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:41.621762 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.621742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:41.736676 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.736639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:41.830571 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.830535 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" Jun 09 14:20:41.974665 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:41.974643 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z"] Jun 09 14:20:41.976812 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:41.976784 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4c8b66_34c8_4ba6_8f94_4e8fdbe0870c.slice/crio-35b33731cf5a5b84ada9d790d99f1649f91c06fb41d71cf1240f7b35ddb8c593 WatchSource:0}: Error finding container 35b33731cf5a5b84ada9d790d99f1649f91c06fb41d71cf1240f7b35ddb8c593: Status 404 returned error can't find the container with id 35b33731cf5a5b84ada9d790d99f1649f91c06fb41d71cf1240f7b35ddb8c593 Jun 09 14:20:42.037077 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.037007 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:42.040767 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.040744 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-zltjm"] Jun 09 14:20:42.043686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.043669 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.045956 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.045924 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 09 14:20:42.046062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.046046 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-j66nh\"" Jun 09 14:20:42.053294 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.053266 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-zltjm"] Jun 09 14:20:42.066112 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.066082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31edc12e-1f61-48f2-b550-628cd0e6b021-cert\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:42.066222 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.066116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe01e53-7e7f-4e81-ad29-ca40fa3af222-tls-certs\") pod \"model-serving-api-5dd488fc48-zltjm\" (UID: \"2fe01e53-7e7f-4e81-ad29-ca40fa3af222\") " pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.066222 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.066166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skgjv\" (UniqueName: \"kubernetes.io/projected/2fe01e53-7e7f-4e81-ad29-ca40fa3af222-kube-api-access-skgjv\") pod \"model-serving-api-5dd488fc48-zltjm\" (UID: \"2fe01e53-7e7f-4e81-ad29-ca40fa3af222\") " pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.068828 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.068806 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31edc12e-1f61-48f2-b550-628cd0e6b021-cert\") pod \"odh-model-controller-6dc94f689f-l8fbt\" (UID: \"31edc12e-1f61-48f2-b550-628cd0e6b021\") " pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:42.116049 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.116021 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:42.166675 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.166638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe01e53-7e7f-4e81-ad29-ca40fa3af222-tls-certs\") pod \"model-serving-api-5dd488fc48-zltjm\" (UID: \"2fe01e53-7e7f-4e81-ad29-ca40fa3af222\") " pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.166790 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.166690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skgjv\" (UniqueName: \"kubernetes.io/projected/2fe01e53-7e7f-4e81-ad29-ca40fa3af222-kube-api-access-skgjv\") pod \"model-serving-api-5dd488fc48-zltjm\" (UID: \"2fe01e53-7e7f-4e81-ad29-ca40fa3af222\") " pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.169111 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.169091 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fe01e53-7e7f-4e81-ad29-ca40fa3af222-tls-certs\") pod \"model-serving-api-5dd488fc48-zltjm\" (UID: \"2fe01e53-7e7f-4e81-ad29-ca40fa3af222\") " pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.174765 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.174741 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgjv\" (UniqueName: \"kubernetes.io/projected/2fe01e53-7e7f-4e81-ad29-ca40fa3af222-kube-api-access-skgjv\") pod \"model-serving-api-5dd488fc48-zltjm\" (UID: \"2fe01e53-7e7f-4e81-ad29-ca40fa3af222\") " pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.303714 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.303651 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:42.355686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.355654 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:42.435046 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.435002 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-l8fbt"] Jun 09 14:20:42.437135 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:42.437112 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31edc12e_1f61_48f2_b550_628cd0e6b021.slice/crio-dfc5137017e1906a3970198d0b22a2dbee13ba1182ea7ef811367425db53fee9 WatchSource:0}: Error finding container dfc5137017e1906a3970198d0b22a2dbee13ba1182ea7ef811367425db53fee9: Status 404 returned error can't find the container with id dfc5137017e1906a3970198d0b22a2dbee13ba1182ea7ef811367425db53fee9 Jun 09 14:20:42.488789 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.488760 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-zltjm"] Jun 09 14:20:42.491587 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:42.491560 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe01e53_7e7f_4e81_ad29_ca40fa3af222.slice/crio-42a415c4277cdf3e33f51ba61732dfe7216ab01f884dbf3082b3f67bea7ace1b WatchSource:0}: Error finding container 42a415c4277cdf3e33f51ba61732dfe7216ab01f884dbf3082b3f67bea7ace1b: Status 404 returned error can't find the container with id 42a415c4277cdf3e33f51ba61732dfe7216ab01f884dbf3082b3f67bea7ace1b Jun 09 14:20:42.894352 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.894313 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" event={"ID":"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c","Type":"ContainerStarted","Data":"35b33731cf5a5b84ada9d790d99f1649f91c06fb41d71cf1240f7b35ddb8c593"} Jun 09 14:20:42.895799 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.895764 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" event={"ID":"2fe01e53-7e7f-4e81-ad29-ca40fa3af222","Type":"ContainerStarted","Data":"42a415c4277cdf3e33f51ba61732dfe7216ab01f884dbf3082b3f67bea7ace1b"} Jun 09 14:20:42.897092 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:42.897069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" event={"ID":"31edc12e-1f61-48f2-b550-628cd0e6b021","Type":"ContainerStarted","Data":"dfc5137017e1906a3970198d0b22a2dbee13ba1182ea7ef811367425db53fee9"} Jun 09 14:20:47.927961 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.927924 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" event={"ID":"31edc12e-1f61-48f2-b550-628cd0e6b021","Type":"ContainerStarted","Data":"8b5e0bd3346ffd5867135aad7e856c1d66be2c0a4bf04ccc577b1b39f43d859d"} Jun 09 14:20:47.928446 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.928032 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:47.929494 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.929463 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" event={"ID":"fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c","Type":"ContainerStarted","Data":"4d99ed16bbd3d8d695fb85ab0748a3d7316c1a1706df210b59bd9a5b01881313"} Jun 09 14:20:47.930765 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.930747 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" event={"ID":"2fe01e53-7e7f-4e81-ad29-ca40fa3af222","Type":"ContainerStarted","Data":"9898412e3989739ad3bffca4840d30fec6549231f1903d200e3c1342dfbe5a9a"} Jun 09 14:20:47.930857 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.930792 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:47.945384 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.945336 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" podStartSLOduration=2.455725211 podStartE2EDuration="6.945323103s" totalImagesPullingTime="4.489597892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:20:47.943734871 +0000 UTC m=+512.590481563" watchObservedRunningTime="2026-06-09 14:20:47.945323103 +0000 UTC m=+512.592069797" Jun 09 14:20:47.959923 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.959873 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6df8999ffb-wff2z" podStartSLOduration=3.026411499 podStartE2EDuration="7.959857728s" totalImagesPullingTime="4.933446229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:20:47.958666615 +0000 UTC m=+512.605413308" watchObservedRunningTime="2026-06-09 14:20:47.959857728 +0000 UTC m=+512.606604421" Jun 09 14:20:47.978227 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:47.978170 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" podStartSLOduration=1.554509628 podStartE2EDuration="5.978150089s" totalImagesPullingTime="4.423640461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:20:47.975929024 +0000 UTC m=+512.622675719" watchObservedRunningTime="2026-06-09 14:20:47.978150089 +0000 UTC m=+512.624896784" Jun 09 14:20:50.359712 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.359675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:50.417765 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.417738 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-bnb54"] Jun 09 14:20:50.420007 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.419990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:50.423061 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.423020 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dhdn8\"" Jun 09 14:20:50.423707 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.423685 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 09 14:20:50.425888 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.425868 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6"] Jun 09 14:20:50.428682 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.428665 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:50.431067 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.431044 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ntxjv\"" Jun 09 14:20:50.431187 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.431144 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 09 14:20:50.431595 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.431563 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-bnb54"] Jun 09 14:20:50.438902 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.438879 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6"] Jun 09 14:20:50.445417 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.445381 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:50.542297 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.542271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44185177-68e6-4347-91e0-f4bf207f810a-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:50.542448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.542334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9w4r\" (UniqueName: \"kubernetes.io/projected/44185177-68e6-4347-91e0-f4bf207f810a-kube-api-access-w9w4r\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:50.542448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.542372 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6b72\" (UniqueName: \"kubernetes.io/projected/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-kube-api-access-x6b72\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:50.542533 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.542449 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-cert\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:50.643621 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.643551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9w4r\" (UniqueName: \"kubernetes.io/projected/44185177-68e6-4347-91e0-f4bf207f810a-kube-api-access-w9w4r\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:50.643621 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.643592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6b72\" (UniqueName: \"kubernetes.io/projected/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-kube-api-access-x6b72\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:50.643762 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.643667 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-cert\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:50.643762 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.643711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44185177-68e6-4347-91e0-f4bf207f810a-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:50.643867 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:50.643853 2568 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 09 14:20:50.643921 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:50.643867 2568 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 09 14:20:50.643921 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:50.643916 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-cert podName:56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0 nodeName:}" failed. No retries permitted until 2026-06-09 14:20:51.143896989 +0000 UTC m=+515.790643667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-cert") pod "kserve-controller-manager-69fc866d76-bnb54" (UID: "56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0") : secret "kserve-webhook-server-cert" not found Jun 09 14:20:50.644007 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:20:50.643948 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44185177-68e6-4347-91e0-f4bf207f810a-cert podName:44185177-68e6-4347-91e0-f4bf207f810a nodeName:}" failed. No retries permitted until 2026-06-09 14:20:51.143930476 +0000 UTC m=+515.790677171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/44185177-68e6-4347-91e0-f4bf207f810a-cert") pod "llmisvc-controller-manager-864bfd7bc9-zppv6" (UID: "44185177-68e6-4347-91e0-f4bf207f810a") : secret "llmisvc-webhook-server-cert" not found Jun 09 14:20:50.661531 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.661505 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6b72\" (UniqueName: \"kubernetes.io/projected/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-kube-api-access-x6b72\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:50.664985 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.664963 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9w4r\" (UniqueName: \"kubernetes.io/projected/44185177-68e6-4347-91e0-f4bf207f810a-kube-api-access-w9w4r\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:50.874121 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:50.874084 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:51.150639 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.150607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-cert\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:51.150856 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.150663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44185177-68e6-4347-91e0-f4bf207f810a-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:51.153273 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.153247 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44185177-68e6-4347-91e0-f4bf207f810a-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-zppv6\" (UID: \"44185177-68e6-4347-91e0-f4bf207f810a\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:51.153376 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.153311 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0-cert\") pod \"kserve-controller-manager-69fc866d76-bnb54\" (UID: \"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0\") " pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:51.158344 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.158320 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:51.331963 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.331928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:51.341064 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.341044 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:51.487735 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.487708 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-bnb54"] Jun 09 14:20:51.490947 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:51.490904 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a1a2b7_48c2_4333_9f78_c5fb5e5d9ce0.slice/crio-4c09e342129d53db219d5438b4d6f6276e28390468a082964d9c4e3ebf49aa95 WatchSource:0}: Error finding container 4c09e342129d53db219d5438b4d6f6276e28390468a082964d9c4e3ebf49aa95: Status 404 returned error can't find the container with id 4c09e342129d53db219d5438b4d6f6276e28390468a082964d9c4e3ebf49aa95 Jun 09 14:20:51.516049 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.515986 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6"] Jun 09 14:20:51.518508 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:51.518480 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod44185177_68e6_4347_91e0_f4bf207f810a.slice/crio-06a5508f3a2bbae85a38c8a553f9c672c683432b9fac72c72fe63b36bfea3dfc WatchSource:0}: Error finding container 06a5508f3a2bbae85a38c8a553f9c672c683432b9fac72c72fe63b36bfea3dfc: Status 404 returned error can't find the container with id 06a5508f3a2bbae85a38c8a553f9c672c683432b9fac72c72fe63b36bfea3dfc Jun 09 14:20:51.662764 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.662737 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:51.780708 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.780687 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:51.859861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.859827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:51.953302 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.953270 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" event={"ID":"44185177-68e6-4347-91e0-f4bf207f810a","Type":"ContainerStarted","Data":"06a5508f3a2bbae85a38c8a553f9c672c683432b9fac72c72fe63b36bfea3dfc"} Jun 09 14:20:51.954340 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:51.954315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" event={"ID":"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0","Type":"ContainerStarted","Data":"4c09e342129d53db219d5438b4d6f6276e28390468a082964d9c4e3ebf49aa95"} Jun 09 14:20:55.975868 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:55.975775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" event={"ID":"56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0","Type":"ContainerStarted","Data":"86b38f1055f4ef2558f2000f4accf612f17b6f3ddaa15fca56546c7d8d494b0e"} Jun 09 14:20:55.976282 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:55.975940 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:20:55.977194 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:55.977164 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" event={"ID":"44185177-68e6-4347-91e0-f4bf207f810a","Type":"ContainerStarted","Data":"c3ac60cc2f216399e87ed167765e377fc13715b60012f2c0c2e9ee80326e4dad"} Jun 09 14:20:55.977339 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:55.977286 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:20:55.992625 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:55.992584 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" podStartSLOduration=2.451052544 podStartE2EDuration="5.992571202s" totalImagesPullingTime="3.541518658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:20:55.9921314 +0000 UTC m=+520.638878091" watchObservedRunningTime="2026-06-09 14:20:55.992571202 +0000 UTC m=+520.639317893" Jun 09 14:20:56.007972 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:56.007933 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" podStartSLOduration=2.495708762 podStartE2EDuration="6.0079208s" totalImagesPullingTime="3.512212038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:20:56.006961575 +0000 UTC m=+520.653708272" watchObservedRunningTime="2026-06-09 14:20:56.0079208 +0000 UTC m=+520.654667524" Jun 09 14:20:56.800910 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:56.800858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:56.900353 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:56.900321 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:57.986807 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:57.986775 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:58.073373 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:58.073350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:58.453586 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:58.453515 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:58.748744 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:58.748717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:58.828822 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:58.828794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:58.939270 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:58.939250 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-zltjm" Jun 09 14:20:58.959841 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:58.959824 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-l8fbt" Jun 09 14:20:59.335802 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.335772 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:20:59.493286 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.493259 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-cr5br"] Jun 09 14:20:59.496525 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.496510 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.498959 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.498935 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 09 14:20:59.499096 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.498935 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-99fjj\"" Jun 09 14:20:59.499096 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.498971 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 09 14:20:59.510643 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.510625 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-cr5br"] Jun 09 14:20:59.623229 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.623167 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkxw4\" (UniqueName: \"kubernetes.io/projected/64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7-kube-api-access-bkxw4\") pod \"servicemesh-operator3-57f65f65fb-cr5br\" (UID: \"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.623229 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.623196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7-operator-config\") pod \"servicemesh-operator3-57f65f65fb-cr5br\" (UID: \"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.724586 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.724552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkxw4\" (UniqueName: \"kubernetes.io/projected/64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7-kube-api-access-bkxw4\") pod \"servicemesh-operator3-57f65f65fb-cr5br\" (UID: \"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.724753 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.724593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7-operator-config\") pod \"servicemesh-operator3-57f65f65fb-cr5br\" (UID: \"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.727471 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.727442 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7-operator-config\") pod \"servicemesh-operator3-57f65f65fb-cr5br\" (UID: \"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.734615 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.734591 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkxw4\" (UniqueName: \"kubernetes.io/projected/64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7-kube-api-access-bkxw4\") pod \"servicemesh-operator3-57f65f65fb-cr5br\" (UID: \"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.806436 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.806394 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:20:59.951505 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.951450 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-cr5br"] Jun 09 14:20:59.954013 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:20:59.953985 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64e43e93_2e5d_4c1c_bdb1_60bbdea15eb7.slice/crio-1483df73d75cc4a1ed4ca327e9b366233fd2eb4e78b280c27ce24018a657dcc7 WatchSource:0}: Error finding container 1483df73d75cc4a1ed4ca327e9b366233fd2eb4e78b280c27ce24018a657dcc7: Status 404 returned error can't find the container with id 1483df73d75cc4a1ed4ca327e9b366233fd2eb4e78b280c27ce24018a657dcc7 Jun 09 14:20:59.997028 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:20:59.997002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" event={"ID":"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7","Type":"ContainerStarted","Data":"1483df73d75cc4a1ed4ca327e9b366233fd2eb4e78b280c27ce24018a657dcc7"} Jun 09 14:21:03.016063 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:03.016015 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" event={"ID":"64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7","Type":"ContainerStarted","Data":"2c33d6f94f1adc6fa8dc410de399cf30a287852200ce00b97d9c57dab29a6b5b"} Jun 09 14:21:03.016629 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:03.016158 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:21:03.053065 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:03.053006 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" podStartSLOduration=1.295324223 podStartE2EDuration="4.052985933s" totalImagesPullingTime="2.75766171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:21:03.050166234 +0000 UTC m=+527.696912965" watchObservedRunningTime="2026-06-09 14:21:03.052985933 +0000 UTC m=+527.699732627" Jun 09 14:21:09.517894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:09.517861 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:09.576923 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:09.576883 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:10.547019 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.546990 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87"] Jun 09 14:21:10.550584 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.550564 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.552940 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.552911 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 09 14:21:10.552940 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.552910 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 09 14:21:10.553093 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.552910 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 09 14:21:10.553093 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.552921 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 09 14:21:10.553093 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.552922 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v2fvc\"" Jun 09 14:21:10.558836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.558812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:10.563061 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.563039 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87"] Jun 09 14:21:10.616916 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.616887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:10.624017 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.623994 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2h7z\" (UniqueName: \"kubernetes.io/projected/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-kube-api-access-m2h7z\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.624131 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.624028 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.624131 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.624047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.624131 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.624067 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.624131 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.624119 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.624275 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.624141 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.624275 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.624184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725639 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725793 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725793 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725704 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725793 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2h7z\" (UniqueName: \"kubernetes.io/projected/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-kube-api-access-m2h7z\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.725953 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.725860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.726289 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.726264 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.728233 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.728205 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.728717 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.728693 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.728776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.728716 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.728776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.728726 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.734187 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.734158 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2h7z\" (UniqueName: \"kubernetes.io/projected/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-kube-api-access-m2h7z\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.734591 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.734570 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c5d6f-2a71-4d36-96cb-76aa5d5205b4-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-2tn87\" (UID: \"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.860173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.860081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:10.990862 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:10.990831 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:11.004165 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:11.004145 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87"] Jun 09 14:21:11.006517 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:21:11.006490 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08c5d6f_2a71_4d36_96cb_76aa5d5205b4.slice/crio-5c976404c82acc7f4ce76656b5e1cbed5e8a343f53de5868dc88befbd7213bab WatchSource:0}: Error finding container 5c976404c82acc7f4ce76656b5e1cbed5e8a343f53de5868dc88befbd7213bab: Status 404 returned error can't find the container with id 5c976404c82acc7f4ce76656b5e1cbed5e8a343f53de5868dc88befbd7213bab Jun 09 14:21:11.052726 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:11.052699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" event={"ID":"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4","Type":"ContainerStarted","Data":"5c976404c82acc7f4ce76656b5e1cbed5e8a343f53de5868dc88befbd7213bab"} Jun 09 14:21:11.092612 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:11.092578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:11.192642 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:11.192560 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:13.141620 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:13.141585 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:13.195154 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:13.195126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:13.273211 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:13.273180 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 14:21:13.273277 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:13.273248 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 09 14:21:13.773932 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:13.773889 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:13.874214 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:13.874159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:14.022303 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.022267 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-cr5br" Jun 09 14:21:14.073505 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.073419 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" event={"ID":"e08c5d6f-2a71-4d36-96cb-76aa5d5205b4","Type":"ContainerStarted","Data":"b6e9298c13f2fc6320162f3ec928ff89a6de003339024318e728361423ca0069"} Jun 09 14:21:14.073675 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.073610 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:14.075298 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.075271 2568 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-2tn87 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 09 14:21:14.075475 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.075323 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" podUID="e08c5d6f-2a71-4d36-96cb-76aa5d5205b4" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 14:21:14.083134 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.083106 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:14.098831 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.098785 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" podStartSLOduration=1.83444476 podStartE2EDuration="4.098770724s" totalImagesPullingTime="2.264325964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:21:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:21:14.096802702 +0000 UTC m=+538.743549396" watchObservedRunningTime="2026-06-09 14:21:14.098770724 +0000 UTC m=+538.745517418" Jun 09 14:21:14.348785 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.348680 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:14.456912 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:14.456885 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:15.080007 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:15.079980 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-2tn87" Jun 09 14:21:18.396265 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.396221 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-986494677-9pr9d"] Jun 09 14:21:18.415986 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.415953 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:18.454025 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.453993 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-986494677-9pr9d"] Jun 09 14:21:18.454179 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.454095 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.456539 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.456513 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 09 14:21:18.456667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.456608 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fdnmf\"" Jun 09 14:21:18.598108 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.598068 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/d3e182c6-611b-415e-bdc6-1cc1bd1b9d45-redirect-config\") pod \"dashboard-redirect-986494677-9pr9d\" (UID: \"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45\") " pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.598289 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.598121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf57s\" (UniqueName: \"kubernetes.io/projected/d3e182c6-611b-415e-bdc6-1cc1bd1b9d45-kube-api-access-rf57s\") pod \"dashboard-redirect-986494677-9pr9d\" (UID: \"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45\") " pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.698845 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.698748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/d3e182c6-611b-415e-bdc6-1cc1bd1b9d45-redirect-config\") pod \"dashboard-redirect-986494677-9pr9d\" (UID: \"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45\") " pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.698845 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.698810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf57s\" (UniqueName: \"kubernetes.io/projected/d3e182c6-611b-415e-bdc6-1cc1bd1b9d45-kube-api-access-rf57s\") pod \"dashboard-redirect-986494677-9pr9d\" (UID: \"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45\") " pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.699456 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.699436 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/d3e182c6-611b-415e-bdc6-1cc1bd1b9d45-redirect-config\") pod \"dashboard-redirect-986494677-9pr9d\" (UID: \"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45\") " pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.706975 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.706953 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf57s\" (UniqueName: \"kubernetes.io/projected/d3e182c6-611b-415e-bdc6-1cc1bd1b9d45-kube-api-access-rf57s\") pod \"dashboard-redirect-986494677-9pr9d\" (UID: \"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45\") " pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.764460 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.764421 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-986494677-9pr9d" Jun 09 14:21:18.912705 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:18.912681 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-986494677-9pr9d"] Jun 09 14:21:18.914988 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:21:18.914963 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e182c6_611b_415e_bdc6_1cc1bd1b9d45.slice/crio-9dfb5db0c28a4afa4f4eb00df16c254ddc56ee01a98b102c0e1979351065bf95 WatchSource:0}: Error finding container 9dfb5db0c28a4afa4f4eb00df16c254ddc56ee01a98b102c0e1979351065bf95: Status 404 returned error can't find the container with id 9dfb5db0c28a4afa4f4eb00df16c254ddc56ee01a98b102c0e1979351065bf95 Jun 09 14:21:19.099650 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:19.099616 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-986494677-9pr9d" event={"ID":"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45","Type":"ContainerStarted","Data":"9dfb5db0c28a4afa4f4eb00df16c254ddc56ee01a98b102c0e1979351065bf95"} Jun 09 14:21:23.119259 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:23.119217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-986494677-9pr9d" event={"ID":"d3e182c6-611b-415e-bdc6-1cc1bd1b9d45","Type":"ContainerStarted","Data":"d739ed5708dc7ff10855534652f5300f8b6cd3ef3c045bbb8e83e6f3d9e88b64"} Jun 09 14:21:23.136191 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:23.136143 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-986494677-9pr9d" podStartSLOduration=1.440305006 podStartE2EDuration="5.136127619s" totalImagesPullingTime="3.695822613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:21:23.134728709 +0000 UTC m=+547.781475403" watchObservedRunningTime="2026-06-09 14:21:23.136127619 +0000 UTC m=+547.782874318" Jun 09 14:21:26.983724 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:26.983694 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-zppv6" Jun 09 14:21:26.987165 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:26.987142 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-bnb54" Jun 09 14:21:37.931817 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:37.931783 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:42.030305 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:42.030271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:42.129799 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:42.129763 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:42.231745 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:42.231704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:42.965211 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:42.965170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:43.165463 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:43.165435 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:48.296304 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.296245 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-779d6b5c65-wxjgq"] Jun 09 14:21:48.301309 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.301283 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.303647 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.303625 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 09 14:21:48.303822 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.303806 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 09 14:21:48.303883 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.303863 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 09 14:21:48.304572 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.304550 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 09 14:21:48.304669 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.304630 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h5lpd\"" Jun 09 14:21:48.304738 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.304717 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 09 14:21:48.308432 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.308414 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 09 14:21:48.312055 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.312032 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779d6b5c65-wxjgq"] Jun 09 14:21:48.353188 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353140 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-config\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.353362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-service-ca\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.353362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-trusted-ca-bundle\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.353362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnngl\" (UniqueName: \"kubernetes.io/projected/26df8fe8-46e8-4f88-be5f-9732eb69b77a-kube-api-access-tnngl\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.353362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353147 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:48.353362 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-oauth-serving-cert\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.353624 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-serving-cert\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.353624 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.353459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-oauth-config\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.409091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.409051 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:48.454840 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.454807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-oauth-config\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455038 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.454881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-config\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455038 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.454920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-service-ca\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455038 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.454940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-trusted-ca-bundle\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455038 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.454998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnngl\" (UniqueName: \"kubernetes.io/projected/26df8fe8-46e8-4f88-be5f-9732eb69b77a-kube-api-access-tnngl\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455263 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.455047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-oauth-serving-cert\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455263 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.455108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-serving-cert\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.455807 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-service-ca\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.455906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.455807 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-config\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.456141 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.456064 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-oauth-serving-cert\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.456359 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.456337 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26df8fe8-46e8-4f88-be5f-9732eb69b77a-trusted-ca-bundle\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.457532 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.457514 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-oauth-config\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.457694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.457678 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26df8fe8-46e8-4f88-be5f-9732eb69b77a-console-serving-cert\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.462600 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.462583 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnngl\" (UniqueName: \"kubernetes.io/projected/26df8fe8-46e8-4f88-be5f-9732eb69b77a-kube-api-access-tnngl\") pod \"console-779d6b5c65-wxjgq\" (UID: \"26df8fe8-46e8-4f88-be5f-9732eb69b77a\") " pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.494640 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.494608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:48.548812 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.548736 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:48.611888 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.611860 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:48.744344 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:48.744318 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779d6b5c65-wxjgq"] Jun 09 14:21:48.746264 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:21:48.746235 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26df8fe8_46e8_4f88_be5f_9732eb69b77a.slice/crio-13d00836fc10b0320d953ae537f43b83aecbd227de30c1d329b9fd515cd0e7a8 WatchSource:0}: Error finding container 13d00836fc10b0320d953ae537f43b83aecbd227de30c1d329b9fd515cd0e7a8: Status 404 returned error can't find the container with id 13d00836fc10b0320d953ae537f43b83aecbd227de30c1d329b9fd515cd0e7a8 Jun 09 14:21:49.095332 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:49.095240 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:49.255682 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:49.255643 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779d6b5c65-wxjgq" event={"ID":"26df8fe8-46e8-4f88-be5f-9732eb69b77a","Type":"ContainerStarted","Data":"ddd3aa47820c0fb3d69bc98d9010e243b155f56c44430315763dd74d7e6bd370"} Jun 09 14:21:49.255880 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:49.255718 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779d6b5c65-wxjgq" event={"ID":"26df8fe8-46e8-4f88-be5f-9732eb69b77a","Type":"ContainerStarted","Data":"13d00836fc10b0320d953ae537f43b83aecbd227de30c1d329b9fd515cd0e7a8"} Jun 09 14:21:49.273186 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:49.273136 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-779d6b5c65-wxjgq" podStartSLOduration=1.273122031 podStartE2EDuration="1.273122031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:21:49.272865474 +0000 UTC m=+573.919612166" watchObservedRunningTime="2026-06-09 14:21:49.273122031 +0000 UTC m=+573.919868724" Jun 09 14:21:54.322417 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:54.322373 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:58.509238 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:58.509205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:58.612504 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:58.612461 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:58.612635 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:58.612536 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:58.617126 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:58.617103 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:59.306140 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:59.306115 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-779d6b5c65-wxjgq" Jun 09 14:21:59.563589 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:59.563514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:21:59.663243 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:21:59.663192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:03.818861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:03.818822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:07.207574 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:07.207542 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:08.023945 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:08.023913 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:10.741041 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.741009 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb"] Jun 09 14:22:10.746045 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.746030 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:10.749667 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.749648 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-q4tf9\"" Jun 09 14:22:10.749972 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.749958 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 14:22:10.750738 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.750724 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 14:22:10.758211 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.758186 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb"] Jun 09 14:22:10.851929 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.851892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkjmk\" (UniqueName: \"kubernetes.io/projected/b3011c70-5142-44ea-b257-fea30b142bee-kube-api-access-xkjmk\") pod \"authorino-operator-75cc8d5cb7-nr2gb\" (UID: \"b3011c70-5142-44ea-b257-fea30b142bee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:10.953034 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.953001 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkjmk\" (UniqueName: \"kubernetes.io/projected/b3011c70-5142-44ea-b257-fea30b142bee-kube-api-access-xkjmk\") pod \"authorino-operator-75cc8d5cb7-nr2gb\" (UID: \"b3011c70-5142-44ea-b257-fea30b142bee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:10.963117 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:10.963091 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkjmk\" (UniqueName: \"kubernetes.io/projected/b3011c70-5142-44ea-b257-fea30b142bee-kube-api-access-xkjmk\") pod \"authorino-operator-75cc8d5cb7-nr2gb\" (UID: \"b3011c70-5142-44ea-b257-fea30b142bee\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:11.057080 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:11.056999 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:11.191542 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:11.191518 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb"] Jun 09 14:22:11.194069 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:22:11.194037 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3011c70_5142_44ea_b257_fea30b142bee.slice/crio-2eed54ca476ee6a151405d093ff96bcd88247ee0829dcb4ac58c8ba73879fc70 WatchSource:0}: Error finding container 2eed54ca476ee6a151405d093ff96bcd88247ee0829dcb4ac58c8ba73879fc70: Status 404 returned error can't find the container with id 2eed54ca476ee6a151405d093ff96bcd88247ee0829dcb4ac58c8ba73879fc70 Jun 09 14:22:11.357460 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:11.357349 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" event={"ID":"b3011c70-5142-44ea-b257-fea30b142bee","Type":"ContainerStarted","Data":"2eed54ca476ee6a151405d093ff96bcd88247ee0829dcb4ac58c8ba73879fc70"} Jun 09 14:22:11.860158 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:11.860118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:12.105766 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:12.105734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:14.083649 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:14.083621 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:14.375806 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:14.375718 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" event={"ID":"b3011c70-5142-44ea-b257-fea30b142bee","Type":"ContainerStarted","Data":"a4bcfa072f8d2aec19a6793996af52f4d06c8f0ca4c565ed3a7a60bc6da0f1af"} Jun 09 14:22:14.375976 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:14.375835 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:14.399829 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:14.399779 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" podStartSLOduration=2.273712764 podStartE2EDuration="4.399761212s" totalImagesPullingTime="2.126048448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:22:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:22:14.397647799 +0000 UTC m=+599.044394494" watchObservedRunningTime="2026-06-09 14:22:14.399761212 +0000 UTC m=+599.046507905" Jun 09 14:22:14.417372 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:14.417337 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:15.078197 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:15.078148 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:15.847573 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:15.847541 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:22:15.848628 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:15.848605 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:22:18.710031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.709994 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq"] Jun 09 14:22:18.713526 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.713509 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:18.715990 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.715969 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tmc76\"" Jun 09 14:22:18.725142 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.725115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6326980f-c6c3-4dcd-b434-1b6a0870f1da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:18.725263 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.725192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krd62\" (UniqueName: \"kubernetes.io/projected/6326980f-c6c3-4dcd-b434-1b6a0870f1da-kube-api-access-krd62\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:18.726279 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.726257 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq"] Jun 09 14:22:18.826072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.826041 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krd62\" (UniqueName: \"kubernetes.io/projected/6326980f-c6c3-4dcd-b434-1b6a0870f1da-kube-api-access-krd62\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:18.826255 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.826113 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6326980f-c6c3-4dcd-b434-1b6a0870f1da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:18.826536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.826519 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6326980f-c6c3-4dcd-b434-1b6a0870f1da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:18.834427 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:18.834377 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krd62\" (UniqueName: \"kubernetes.io/projected/6326980f-c6c3-4dcd-b434-1b6a0870f1da-kube-api-access-krd62\") pod \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:19.023788 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:19.023753 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:19.165360 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:19.165336 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq"] Jun 09 14:22:19.167171 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:22:19.167147 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6326980f_c6c3_4dcd_b434_1b6a0870f1da.slice/crio-69d4feac5d79630f72763fe878a2d511ccd51ff4499e699b75abcc5745ee8159 WatchSource:0}: Error finding container 69d4feac5d79630f72763fe878a2d511ccd51ff4499e699b75abcc5745ee8159: Status 404 returned error can't find the container with id 69d4feac5d79630f72763fe878a2d511ccd51ff4499e699b75abcc5745ee8159 Jun 09 14:22:19.402715 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:19.402631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" event={"ID":"6326980f-c6c3-4dcd-b434-1b6a0870f1da","Type":"ContainerStarted","Data":"69d4feac5d79630f72763fe878a2d511ccd51ff4499e699b75abcc5745ee8159"} Jun 09 14:22:20.865694 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:20.865658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:25.019478 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:25.019434 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:25.087637 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:25.087603 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:25.384163 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:25.384078 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-nr2gb" Jun 09 14:22:25.434774 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:25.434745 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:25.531302 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:25.531269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:28.453441 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:28.453322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" event={"ID":"6326980f-c6c3-4dcd-b434-1b6a0870f1da","Type":"ContainerStarted","Data":"48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92"} Jun 09 14:22:28.453441 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:28.453423 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:28.473896 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:28.473845 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" podStartSLOduration=1.4914105389999999 podStartE2EDuration="10.473829332s" totalImagesPullingTime="8.982418793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:22:28.470930374 +0000 UTC m=+613.117677067" watchObservedRunningTime="2026-06-09 14:22:28.473829332 +0000 UTC m=+613.120576025" Jun 09 14:22:29.019836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:29.019801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:29.120258 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:29.120221 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:30.915224 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:30.915192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:31.530147 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:31.530116 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:32.620111 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:32.620071 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:32.683549 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:32.683511 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:39.167779 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:39.167738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:39.266713 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:39.266679 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:39.461330 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:39.461244 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.191148 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.191114 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq"] Jun 09 14:22:41.191635 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.191311 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" containerName="manager" containerID="cri-o://48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92" gracePeriod=2 Jun 09 14:22:41.200989 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.200959 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq"] Jun 09 14:22:41.211916 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:22:41.211879 2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6326980f-c6c3-4dcd-b434-1b6a0870f1da\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-82npq\": pods \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.217680 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.217653 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r"] Jun 09 14:22:41.218053 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.218041 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" containerName="manager" Jun 09 14:22:41.218139 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.218054 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" containerName="manager" Jun 09 14:22:41.218139 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.218133 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" containerName="manager" Jun 09 14:22:41.220883 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.220863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.238044 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.238012 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r"] Jun 09 14:22:41.260556 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:22:41.260512 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" is forbidden: User \"system:node:ip-10-0-128-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-214.ec2.internal' and this object" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.343478 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.343442 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj9p8\" (UniqueName: \"kubernetes.io/projected/9b496d56-06c2-4659-9be0-41e2be5f33ef-kube-api-access-fj9p8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k8v2r\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.343622 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.343595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b496d56-06c2-4659-9be0-41e2be5f33ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k8v2r\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.435137 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.435116 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.440024 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:22:41.440001 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" is forbidden: User \"system:node:ip-10-0-128-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-214.ec2.internal' and this object" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.444490 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.444449 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj9p8\" (UniqueName: \"kubernetes.io/projected/9b496d56-06c2-4659-9be0-41e2be5f33ef-kube-api-access-fj9p8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k8v2r\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.444567 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.444551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b496d56-06c2-4659-9be0-41e2be5f33ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k8v2r\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.444918 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.444903 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b496d56-06c2-4659-9be0-41e2be5f33ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k8v2r\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.455594 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.455564 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj9p8\" (UniqueName: \"kubernetes.io/projected/9b496d56-06c2-4659-9be0-41e2be5f33ef-kube-api-access-fj9p8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k8v2r\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.534587 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.534549 2568 generic.go:356] "Generic (PLEG): container finished" podID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" containerID="48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92" exitCode=0 Jun 09 14:22:41.534778 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.534600 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.534778 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.534619 2568 scope.go:117] "RemoveContainer" containerID="48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92" Jun 09 14:22:41.537368 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:22:41.537339 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" is forbidden: User \"system:node:ip-10-0-128-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-214.ec2.internal' and this object" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.543219 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.543201 2568 scope.go:117] "RemoveContainer" containerID="48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92" Jun 09 14:22:41.543558 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:22:41.543531 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92\": container with ID starting with 48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92 not found: ID does not exist" containerID="48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92" Jun 09 14:22:41.543650 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.543558 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92"} err="failed to get container status \"48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92\": rpc error: code = NotFound desc = could not find container \"48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92\": container with ID starting with 48e601df47ba9d23cd530869f9255a8037d93fc6ea385e3a27bddcdbe23e0e92 not found: ID does not exist" Jun 09 14:22:41.545836 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.545819 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6326980f-c6c3-4dcd-b434-1b6a0870f1da-extensions-socket-volume\") pod \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " Jun 09 14:22:41.545945 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.545933 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krd62\" (UniqueName: \"kubernetes.io/projected/6326980f-c6c3-4dcd-b434-1b6a0870f1da-kube-api-access-krd62\") pod \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\" (UID: \"6326980f-c6c3-4dcd-b434-1b6a0870f1da\") " Jun 09 14:22:41.546279 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.546256 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6326980f-c6c3-4dcd-b434-1b6a0870f1da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6326980f-c6c3-4dcd-b434-1b6a0870f1da" (UID: "6326980f-c6c3-4dcd-b434-1b6a0870f1da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 14:22:41.548071 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.548046 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6326980f-c6c3-4dcd-b434-1b6a0870f1da-kube-api-access-krd62" (OuterVolumeSpecName: "kube-api-access-krd62") pod "6326980f-c6c3-4dcd-b434-1b6a0870f1da" (UID: "6326980f-c6c3-4dcd-b434-1b6a0870f1da"). InnerVolumeSpecName "kube-api-access-krd62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:22:41.581393 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.581350 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:41.655041 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.649897 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krd62\" (UniqueName: \"kubernetes.io/projected/6326980f-c6c3-4dcd-b434-1b6a0870f1da-kube-api-access-krd62\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:22:41.655041 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.649932 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6326980f-c6c3-4dcd-b434-1b6a0870f1da-extensions-socket-volume\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:22:41.745319 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.745283 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r"] Jun 09 14:22:41.749859 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:22:41.749832 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b496d56_06c2_4659_9be0_41e2be5f33ef.slice/crio-7478d529c1ee66bbc71a33eafd3abcd55c42c8a73101e9a779e8268778fd2404 WatchSource:0}: Error finding container 7478d529c1ee66bbc71a33eafd3abcd55c42c8a73101e9a779e8268778fd2404: Status 404 returned error can't find the container with id 7478d529c1ee66bbc71a33eafd3abcd55c42c8a73101e9a779e8268778fd2404 Jun 09 14:22:41.844971 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:22:41.844937 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-82npq\" is forbidden: User \"system:node:ip-10-0-128-214.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-214.ec2.internal' and this object" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-82npq" Jun 09 14:22:41.877570 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:41.877538 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6326980f-c6c3-4dcd-b434-1b6a0870f1da" path="/var/lib/kubelet/pods/6326980f-c6c3-4dcd-b434-1b6a0870f1da/volumes" Jun 09 14:22:42.542073 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.542040 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" event={"ID":"9b496d56-06c2-4659-9be0-41e2be5f33ef","Type":"ContainerStarted","Data":"1ff6281da005cf6c7ef719dd9b9d2dda976b232dd956c86c02b3a0de358f6dcb"} Jun 09 14:22:42.542533 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.542099 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" event={"ID":"9b496d56-06c2-4659-9be0-41e2be5f33ef","Type":"ContainerStarted","Data":"7478d529c1ee66bbc71a33eafd3abcd55c42c8a73101e9a779e8268778fd2404"} Jun 09 14:22:42.542533 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.542126 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:42.565686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.565628 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" podStartSLOduration=1.565609728 podStartE2EDuration="1.565609728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:22:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:22:42.563234106 +0000 UTC m=+627.209980799" watchObservedRunningTime="2026-06-09 14:22:42.565609728 +0000 UTC m=+627.212356420" Jun 09 14:22:42.613737 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.613702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:42.713134 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.713098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:42.814691 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:42.814610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:45.045163 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:45.045122 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:45.110312 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:45.110264 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:51.405049 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:51.405011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:51.501047 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:51.501013 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:22:53.550079 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:53.550043 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:57.452325 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.452286 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r"] Jun 09 14:22:57.452882 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.452555 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" podUID="9b496d56-06c2-4659-9be0-41e2be5f33ef" containerName="manager" containerID="cri-o://1ff6281da005cf6c7ef719dd9b9d2dda976b232dd956c86c02b3a0de358f6dcb" gracePeriod=10 Jun 09 14:22:57.622111 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.622067 2568 generic.go:356] "Generic (PLEG): container finished" podID="9b496d56-06c2-4659-9be0-41e2be5f33ef" containerID="1ff6281da005cf6c7ef719dd9b9d2dda976b232dd956c86c02b3a0de358f6dcb" exitCode=0 Jun 09 14:22:57.622111 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.622114 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" event={"ID":"9b496d56-06c2-4659-9be0-41e2be5f33ef","Type":"ContainerDied","Data":"1ff6281da005cf6c7ef719dd9b9d2dda976b232dd956c86c02b3a0de358f6dcb"} Jun 09 14:22:57.696015 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.695983 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7"] Jun 09 14:22:57.699621 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.699600 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:57.710460 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.710372 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7"] Jun 09 14:22:57.712236 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.712216 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:57.796015 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.795980 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj9p8\" (UniqueName: \"kubernetes.io/projected/9b496d56-06c2-4659-9be0-41e2be5f33ef-kube-api-access-fj9p8\") pod \"9b496d56-06c2-4659-9be0-41e2be5f33ef\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " Jun 09 14:22:57.796215 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.796053 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b496d56-06c2-4659-9be0-41e2be5f33ef-extensions-socket-volume\") pod \"9b496d56-06c2-4659-9be0-41e2be5f33ef\" (UID: \"9b496d56-06c2-4659-9be0-41e2be5f33ef\") " Jun 09 14:22:57.796215 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.796127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mn4p\" (UniqueName: \"kubernetes.io/projected/c6119b4f-8267-44a9-a8d4-b3c22544f495-kube-api-access-8mn4p\") pod \"kuadrant-operator-controller-manager-86878c769f-56hq7\" (UID: \"c6119b4f-8267-44a9-a8d4-b3c22544f495\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:57.796215 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.796159 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6119b4f-8267-44a9-a8d4-b3c22544f495-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-56hq7\" (UID: \"c6119b4f-8267-44a9-a8d4-b3c22544f495\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:57.796524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.796500 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b496d56-06c2-4659-9be0-41e2be5f33ef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9b496d56-06c2-4659-9be0-41e2be5f33ef" (UID: "9b496d56-06c2-4659-9be0-41e2be5f33ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 14:22:57.798257 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.798227 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b496d56-06c2-4659-9be0-41e2be5f33ef-kube-api-access-fj9p8" (OuterVolumeSpecName: "kube-api-access-fj9p8") pod "9b496d56-06c2-4659-9be0-41e2be5f33ef" (UID: "9b496d56-06c2-4659-9be0-41e2be5f33ef"). InnerVolumeSpecName "kube-api-access-fj9p8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:22:57.897594 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.897559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mn4p\" (UniqueName: \"kubernetes.io/projected/c6119b4f-8267-44a9-a8d4-b3c22544f495-kube-api-access-8mn4p\") pod \"kuadrant-operator-controller-manager-86878c769f-56hq7\" (UID: \"c6119b4f-8267-44a9-a8d4-b3c22544f495\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:57.897777 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.897606 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6119b4f-8267-44a9-a8d4-b3c22544f495-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-56hq7\" (UID: \"c6119b4f-8267-44a9-a8d4-b3c22544f495\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:57.897777 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.897656 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj9p8\" (UniqueName: \"kubernetes.io/projected/9b496d56-06c2-4659-9be0-41e2be5f33ef-kube-api-access-fj9p8\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:22:57.897777 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.897666 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b496d56-06c2-4659-9be0-41e2be5f33ef-extensions-socket-volume\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:22:57.897959 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.897943 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6119b4f-8267-44a9-a8d4-b3c22544f495-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-56hq7\" (UID: \"c6119b4f-8267-44a9-a8d4-b3c22544f495\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:57.912317 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:57.912280 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mn4p\" (UniqueName: \"kubernetes.io/projected/c6119b4f-8267-44a9-a8d4-b3c22544f495-kube-api-access-8mn4p\") pod \"kuadrant-operator-controller-manager-86878c769f-56hq7\" (UID: \"c6119b4f-8267-44a9-a8d4-b3c22544f495\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:58.021060 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.021021 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:58.159120 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.159086 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7"] Jun 09 14:22:58.162759 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:22:58.162732 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6119b4f_8267_44a9_a8d4_b3c22544f495.slice/crio-01a5388004b37427ab55feae6d51fc30e90e031eabf10f3cad328f0a167418dc WatchSource:0}: Error finding container 01a5388004b37427ab55feae6d51fc30e90e031eabf10f3cad328f0a167418dc: Status 404 returned error can't find the container with id 01a5388004b37427ab55feae6d51fc30e90e031eabf10f3cad328f0a167418dc Jun 09 14:22:58.629545 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.629511 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" event={"ID":"c6119b4f-8267-44a9-a8d4-b3c22544f495","Type":"ContainerStarted","Data":"624da1e0161f8b23dd13d0a26ca7ea979f3d457fafe60040bd9278e3328b186c"} Jun 09 14:22:58.630029 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.629584 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:22:58.630029 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.629612 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" event={"ID":"c6119b4f-8267-44a9-a8d4-b3c22544f495","Type":"ContainerStarted","Data":"01a5388004b37427ab55feae6d51fc30e90e031eabf10f3cad328f0a167418dc"} Jun 09 14:22:58.630861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.630840 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" event={"ID":"9b496d56-06c2-4659-9be0-41e2be5f33ef","Type":"ContainerDied","Data":"7478d529c1ee66bbc71a33eafd3abcd55c42c8a73101e9a779e8268778fd2404"} Jun 09 14:22:58.630947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.630885 2568 scope.go:117] "RemoveContainer" containerID="1ff6281da005cf6c7ef719dd9b9d2dda976b232dd956c86c02b3a0de358f6dcb" Jun 09 14:22:58.630947 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.630848 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r" Jun 09 14:22:58.650188 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.650141 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" podStartSLOduration=1.650127329 podStartE2EDuration="1.650127329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:22:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:22:58.647540124 +0000 UTC m=+643.294286842" watchObservedRunningTime="2026-06-09 14:22:58.650127329 +0000 UTC m=+643.296874087" Jun 09 14:22:58.667561 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.667527 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r"] Jun 09 14:22:58.674356 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:58.674328 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k8v2r"] Jun 09 14:22:59.877986 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:22:59.877946 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b496d56-06c2-4659-9be0-41e2be5f33ef" path="/var/lib/kubelet/pods/9b496d56-06c2-4659-9be0-41e2be5f33ef/volumes" Jun 09 14:23:00.998494 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:00.998460 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:01.095918 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:01.095883 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:05.398820 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:05.398781 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:05.452338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:05.452300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:05.600103 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:05.600075 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:05.701624 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:05.701539 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:07.455636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:07.455606 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:07.556190 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:07.556155 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:09.639624 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:09.639590 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-56hq7" Jun 09 14:23:12.752984 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:12.752943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:12.853171 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:12.853117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:13.766612 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:13.766553 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:13.815156 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:13.815117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:15.643536 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:15.643498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:15.745027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:15.744976 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:17.601955 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:17.601922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:29.089031 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:29.088991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:29.187302 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:29.187253 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:29.284744 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:29.284702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:29.896330 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:29.896300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:29.990277 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:29.990243 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:49.462019 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:49.461984 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:57.315224 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:57.315143 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:23:57.415926 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:23:57.415887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:02.061095 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:02.061056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:02.154975 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:02.154933 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:02.287207 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:02.287176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:27.083802 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:27.083770 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:27.182357 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:27.182321 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:56.570182 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:56.570147 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:24:56.833923 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:24:56.833838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:01.761209 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.761180 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:01.776537 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.776510 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84d4fd9b44-g96rj"] Jun 09 14:25:01.776934 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.776914 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b496d56-06c2-4659-9be0-41e2be5f33ef" containerName="manager" Jun 09 14:25:01.776934 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.776931 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b496d56-06c2-4659-9be0-41e2be5f33ef" containerName="manager" Jun 09 14:25:01.777101 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.777019 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b496d56-06c2-4659-9be0-41e2be5f33ef" containerName="manager" Jun 09 14:25:01.779531 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.779512 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:01.781888 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.781864 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 14:25:01.782014 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.781973 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 14:25:01.782072 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.782013 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mljdg\"" Jun 09 14:25:01.789356 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.789336 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d4fd9b44-g96rj"] Jun 09 14:25:01.857425 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.857365 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:01.906727 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.906689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:01.906891 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.906801 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x428s\" (UniqueName: \"kubernetes.io/projected/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-kube-api-access-x428s\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:01.925375 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.925332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:01.948810 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.948782 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-t5fdz"] Jun 09 14:25:01.951714 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.951698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:01.962524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:01.962501 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-t5fdz"] Jun 09 14:25:02.007826 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.007802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:02.007950 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.007854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x428s\" (UniqueName: \"kubernetes.io/projected/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-kube-api-access-x428s\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:02.007950 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:25:02.007941 2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 14:25:02.008022 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:25:02.008012 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert podName:aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b nodeName:}" failed. No retries permitted until 2026-06-09 14:25:02.507995564 +0000 UTC m=+767.154742235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert") pod "maas-controller-84d4fd9b44-g96rj" (UID: "aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b") : secret "maas-controller-webhook-cert" not found Jun 09 14:25:02.016176 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.016118 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x428s\" (UniqueName: \"kubernetes.io/projected/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-kube-api-access-x428s\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:02.109009 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.108980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbvp8\" (UniqueName: \"kubernetes.io/projected/c12bfd07-e56f-4840-958e-b595012b4acd-kube-api-access-xbvp8\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.109155 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.109105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.210317 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.210267 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.210524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.210352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbvp8\" (UniqueName: \"kubernetes.io/projected/c12bfd07-e56f-4840-958e-b595012b4acd-kube-api-access-xbvp8\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.210524 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:25:02.210447 2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 14:25:02.210524 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:25:02.210512 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert podName:c12bfd07-e56f-4840-958e-b595012b4acd nodeName:}" failed. No retries permitted until 2026-06-09 14:25:02.710497564 +0000 UTC m=+767.357244235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert") pod "maas-controller-585c5cbb6c-t5fdz" (UID: "c12bfd07-e56f-4840-958e-b595012b4acd") : secret "maas-controller-webhook-cert" not found Jun 09 14:25:02.221094 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.221070 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbvp8\" (UniqueName: \"kubernetes.io/projected/c12bfd07-e56f-4840-958e-b595012b4acd-kube-api-access-xbvp8\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.306673 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.306594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:02.514020 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.513984 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:02.517210 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.517179 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert\") pod \"maas-controller-84d4fd9b44-g96rj\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:02.691577 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.691499 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:02.716025 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.715996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.718541 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.718517 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert\") pod \"maas-controller-585c5cbb6c-t5fdz\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.831516 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.831481 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84d4fd9b44-g96rj"] Jun 09 14:25:02.833794 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:25:02.833765 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2c9d23_5dad_4b3e_b1d6_f8c6ff1a790b.slice/crio-6b4da0bfcddc92897435bd7a251803de0758fcc0f7b367d698ac5256e66bdfa6 WatchSource:0}: Error finding container 6b4da0bfcddc92897435bd7a251803de0758fcc0f7b367d698ac5256e66bdfa6: Status 404 returned error can't find the container with id 6b4da0bfcddc92897435bd7a251803de0758fcc0f7b367d698ac5256e66bdfa6 Jun 09 14:25:02.834976 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.834959 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 14:25:02.863071 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.863045 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:02.994493 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:02.994467 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-t5fdz"] Jun 09 14:25:02.996969 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:25:02.996944 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12bfd07_e56f_4840_958e_b595012b4acd.slice/crio-a23417e109a290e5bda2f1a9a1b7ca6ce4b5d48ed688cb0dc71df81516e5fdfe WatchSource:0}: Error finding container a23417e109a290e5bda2f1a9a1b7ca6ce4b5d48ed688cb0dc71df81516e5fdfe: Status 404 returned error can't find the container with id a23417e109a290e5bda2f1a9a1b7ca6ce4b5d48ed688cb0dc71df81516e5fdfe Jun 09 14:25:03.270719 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:03.270683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" event={"ID":"c12bfd07-e56f-4840-958e-b595012b4acd","Type":"ContainerStarted","Data":"a23417e109a290e5bda2f1a9a1b7ca6ce4b5d48ed688cb0dc71df81516e5fdfe"} Jun 09 14:25:03.271902 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:03.271877 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" event={"ID":"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b","Type":"ContainerStarted","Data":"6b4da0bfcddc92897435bd7a251803de0758fcc0f7b367d698ac5256e66bdfa6"} Jun 09 14:25:05.687574 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:05.687500 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:05.776134 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:05.776099 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:06.294324 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:06.294282 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" event={"ID":"c12bfd07-e56f-4840-958e-b595012b4acd","Type":"ContainerStarted","Data":"bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91"} Jun 09 14:25:06.294555 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:06.294350 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:06.295907 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:06.295881 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" event={"ID":"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b","Type":"ContainerStarted","Data":"68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773"} Jun 09 14:25:06.296047 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:06.295932 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:06.312118 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:06.312071 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" podStartSLOduration=2.827742055 podStartE2EDuration="5.312058795s" totalImagesPullingTime="2.48431674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:25:06.30966684 +0000 UTC m=+770.956413533" watchObservedRunningTime="2026-06-09 14:25:06.312058795 +0000 UTC m=+770.958805551" Jun 09 14:25:06.324755 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:06.324711 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" podStartSLOduration=2.6846634590000003 podStartE2EDuration="5.324698241s" totalImagesPullingTime="2.640034782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:25:06.324330398 +0000 UTC m=+770.971077092" watchObservedRunningTime="2026-06-09 14:25:06.324698241 +0000 UTC m=+770.971444935" Jun 09 14:25:17.305922 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.305888 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:25:17.306321 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.306063 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:17.366032 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.365998 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d4fd9b44-g96rj"] Jun 09 14:25:17.366460 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.366213 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" podUID="aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" containerName="manager" containerID="cri-o://68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773" gracePeriod=30 Jun 09 14:25:17.620151 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.620127 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:17.650563 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.650536 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x428s\" (UniqueName: \"kubernetes.io/projected/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-kube-api-access-x428s\") pod \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " Jun 09 14:25:17.650701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.650591 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert\") pod \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\" (UID: \"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b\") " Jun 09 14:25:17.652935 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.652907 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" (UID: "aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:25:17.653091 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.653008 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-kube-api-access-x428s" (OuterVolumeSpecName: "kube-api-access-x428s") pod "aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" (UID: "aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b"). InnerVolumeSpecName "kube-api-access-x428s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:25:17.751728 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.751688 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x428s\" (UniqueName: \"kubernetes.io/projected/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-kube-api-access-x428s\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:25:17.751728 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:17.751723 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b-webhook-cert\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:25:18.358045 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.358012 2568 generic.go:356] "Generic (PLEG): container finished" podID="aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" containerID="68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773" exitCode=0 Jun 09 14:25:18.358045 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.358057 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" Jun 09 14:25:18.358593 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.358098 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" event={"ID":"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b","Type":"ContainerDied","Data":"68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773"} Jun 09 14:25:18.358593 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.358150 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84d4fd9b44-g96rj" event={"ID":"aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b","Type":"ContainerDied","Data":"6b4da0bfcddc92897435bd7a251803de0758fcc0f7b367d698ac5256e66bdfa6"} Jun 09 14:25:18.358593 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.358177 2568 scope.go:117] "RemoveContainer" containerID="68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773" Jun 09 14:25:18.367450 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.367252 2568 scope.go:117] "RemoveContainer" containerID="68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773" Jun 09 14:25:18.367634 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:25:18.367613 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773\": container with ID starting with 68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773 not found: ID does not exist" containerID="68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773" Jun 09 14:25:18.367679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.367644 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773"} err="failed to get container status \"68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773\": rpc error: code = NotFound desc = could not find container \"68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773\": container with ID starting with 68073d1d51ea79b8b202d3ffb5fd4d1c5a686170418f4afa992b79ef67224773 not found: ID does not exist" Jun 09 14:25:18.373862 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.373836 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84d4fd9b44-g96rj"] Jun 09 14:25:18.377862 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:18.377836 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84d4fd9b44-g96rj"] Jun 09 14:25:19.877527 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:19.877495 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" path="/var/lib/kubelet/pods/aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b/volumes" Jun 09 14:25:38.571173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.571091 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:38.630139 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.630106 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6c8bd7b74c-nwhpz"] Jun 09 14:25:38.630644 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.630626 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" containerName="manager" Jun 09 14:25:38.630710 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.630646 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" containerName="manager" Jun 09 14:25:38.630746 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.630739 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa2c9d23-5dad-4b3e-b1d6-f8c6ff1a790b" containerName="manager" Jun 09 14:25:38.633106 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.633090 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.635576 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.635530 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 09 14:25:38.635976 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.635958 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-n4xdg\"" Jun 09 14:25:38.643977 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.643956 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c8bd7b74c-nwhpz"] Jun 09 14:25:38.693235 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.693198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:38.731589 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.731559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdnhn\" (UniqueName: \"kubernetes.io/projected/88138061-245f-49f5-ae80-3007e57a8eb3-kube-api-access-mdnhn\") pod \"maas-api-6c8bd7b74c-nwhpz\" (UID: \"88138061-245f-49f5-ae80-3007e57a8eb3\") " pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.731702 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.731605 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88138061-245f-49f5-ae80-3007e57a8eb3-maas-api-tls\") pod \"maas-api-6c8bd7b74c-nwhpz\" (UID: \"88138061-245f-49f5-ae80-3007e57a8eb3\") " pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.832686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.832608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdnhn\" (UniqueName: \"kubernetes.io/projected/88138061-245f-49f5-ae80-3007e57a8eb3-kube-api-access-mdnhn\") pod \"maas-api-6c8bd7b74c-nwhpz\" (UID: \"88138061-245f-49f5-ae80-3007e57a8eb3\") " pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.832686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.832667 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88138061-245f-49f5-ae80-3007e57a8eb3-maas-api-tls\") pod \"maas-api-6c8bd7b74c-nwhpz\" (UID: \"88138061-245f-49f5-ae80-3007e57a8eb3\") " pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.835361 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.835330 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88138061-245f-49f5-ae80-3007e57a8eb3-maas-api-tls\") pod \"maas-api-6c8bd7b74c-nwhpz\" (UID: \"88138061-245f-49f5-ae80-3007e57a8eb3\") " pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.840132 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.840108 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdnhn\" (UniqueName: \"kubernetes.io/projected/88138061-245f-49f5-ae80-3007e57a8eb3-kube-api-access-mdnhn\") pod \"maas-api-6c8bd7b74c-nwhpz\" (UID: \"88138061-245f-49f5-ae80-3007e57a8eb3\") " pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.944695 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.944665 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:38.990418 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:38.990369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:39.088806 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.088722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:25:39.090214 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.090191 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-p9wg6"] Jun 09 14:25:39.101385 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:25:39.101347 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88138061_245f_49f5_ae80_3007e57a8eb3.slice/crio-59faca07eebf50df640187479d7f25b1b44b2897aa698b8db689260029c47fca WatchSource:0}: Error finding container 59faca07eebf50df640187479d7f25b1b44b2897aa698b8db689260029c47fca: Status 404 returned error can't find the container with id 59faca07eebf50df640187479d7f25b1b44b2897aa698b8db689260029c47fca Jun 09 14:25:39.112371 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.112349 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c8bd7b74c-nwhpz"] Jun 09 14:25:39.112503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.112428 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-p9wg6"] Jun 09 14:25:39.112503 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.112468 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:25:39.115157 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.115138 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7n8tc\"" Jun 09 14:25:39.116174 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.116155 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 09 14:25:39.120758 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.120735 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-2rj9c"] Jun 09 14:25:39.136562 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.136542 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-2rj9c"] Jun 09 14:25:39.136679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.136637 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:39.236094 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.236064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5zkx\" (UniqueName: \"kubernetes.io/projected/a5730f4a-2886-409d-b625-2e92ee477f88-kube-api-access-l5zkx\") pod \"payload-pre-processing-67d7689b65-p9wg6\" (UID: \"a5730f4a-2886-409d-b625-2e92ee477f88\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:25:39.236274 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.236119 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwdlg\" (UniqueName: \"kubernetes.io/projected/ff0c800c-89d1-42ab-9b21-91da52b54347-kube-api-access-wwdlg\") pod \"payload-processing-7c6c859784-2rj9c\" (UID: \"ff0c800c-89d1-42ab-9b21-91da52b54347\") " pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:39.337635 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.337598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5zkx\" (UniqueName: \"kubernetes.io/projected/a5730f4a-2886-409d-b625-2e92ee477f88-kube-api-access-l5zkx\") pod \"payload-pre-processing-67d7689b65-p9wg6\" (UID: \"a5730f4a-2886-409d-b625-2e92ee477f88\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:25:39.337824 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.337682 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwdlg\" (UniqueName: \"kubernetes.io/projected/ff0c800c-89d1-42ab-9b21-91da52b54347-kube-api-access-wwdlg\") pod \"payload-processing-7c6c859784-2rj9c\" (UID: \"ff0c800c-89d1-42ab-9b21-91da52b54347\") " pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:39.346689 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.346629 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwdlg\" (UniqueName: \"kubernetes.io/projected/ff0c800c-89d1-42ab-9b21-91da52b54347-kube-api-access-wwdlg\") pod \"payload-processing-7c6c859784-2rj9c\" (UID: \"ff0c800c-89d1-42ab-9b21-91da52b54347\") " pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:39.346806 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.346738 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5zkx\" (UniqueName: \"kubernetes.io/projected/a5730f4a-2886-409d-b625-2e92ee477f88-kube-api-access-l5zkx\") pod \"payload-pre-processing-67d7689b65-p9wg6\" (UID: \"a5730f4a-2886-409d-b625-2e92ee477f88\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:25:39.425117 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.425083 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:25:39.447488 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.447462 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:39.466547 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.466519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" event={"ID":"88138061-245f-49f5-ae80-3007e57a8eb3","Type":"ContainerStarted","Data":"59faca07eebf50df640187479d7f25b1b44b2897aa698b8db689260029c47fca"} Jun 09 14:25:39.592928 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.592903 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-p9wg6"] Jun 09 14:25:39.595844 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:25:39.595813 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5730f4a_2886_409d_b625_2e92ee477f88.slice/crio-b2501484bc936925548f0873a55c2b62128c10585c09558b5c26e31fb40fcb3e WatchSource:0}: Error finding container b2501484bc936925548f0873a55c2b62128c10585c09558b5c26e31fb40fcb3e: Status 404 returned error can't find the container with id b2501484bc936925548f0873a55c2b62128c10585c09558b5c26e31fb40fcb3e Jun 09 14:25:39.619325 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:39.619300 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-2rj9c"] Jun 09 14:25:39.621332 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:25:39.621308 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0c800c_89d1_42ab_9b21_91da52b54347.slice/crio-2029b4d1a1c78872e634d66b8ff85429705f149f9181ca8aa1531b9b3ac3f689 WatchSource:0}: Error finding container 2029b4d1a1c78872e634d66b8ff85429705f149f9181ca8aa1531b9b3ac3f689: Status 404 returned error can't find the container with id 2029b4d1a1c78872e634d66b8ff85429705f149f9181ca8aa1531b9b3ac3f689 Jun 09 14:25:40.472956 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:40.472909 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerStarted","Data":"2029b4d1a1c78872e634d66b8ff85429705f149f9181ca8aa1531b9b3ac3f689"} Jun 09 14:25:40.474252 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:40.474226 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" event={"ID":"a5730f4a-2886-409d-b625-2e92ee477f88","Type":"ContainerStarted","Data":"b2501484bc936925548f0873a55c2b62128c10585c09558b5c26e31fb40fcb3e"} Jun 09 14:25:42.486601 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.486565 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" event={"ID":"88138061-245f-49f5-ae80-3007e57a8eb3","Type":"ContainerStarted","Data":"b262d0abdb11a65cd93766404e194c407bc92576fb005e92e2d2a4c7e8622e49"} Jun 09 14:25:42.487025 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.486631 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:42.488186 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.488154 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerStarted","Data":"680dae60a20602c16cb512e7df44d119d3141e8fe64db4a9d2a5ee175674d935"} Jun 09 14:25:42.488324 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.488219 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:42.489612 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.489590 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" event={"ID":"a5730f4a-2886-409d-b625-2e92ee477f88","Type":"ContainerStarted","Data":"a96dd6d2ef5316a06d5b694cd6eeb15893d8c71d0a8c2585e6806d5aa9f719fe"} Jun 09 14:25:42.489765 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.489752 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:25:42.503783 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.503742 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" podStartSLOduration=2.148858666 podStartE2EDuration="4.503729766s" totalImagesPullingTime="2.3548711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:25:42.501417979 +0000 UTC m=+807.148164671" watchObservedRunningTime="2026-06-09 14:25:42.503729766 +0000 UTC m=+807.150476459" Jun 09 14:25:42.516998 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.516962 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" podStartSLOduration=1.693847763 podStartE2EDuration="3.516951108s" totalImagesPullingTime="1.823103345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:25:42.516516647 +0000 UTC m=+807.163263341" watchObservedRunningTime="2026-06-09 14:25:42.516951108 +0000 UTC m=+807.163697841" Jun 09 14:25:42.531861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:42.531821 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podStartSLOduration=1.731064857 podStartE2EDuration="3.531809501s" totalImagesPullingTime="1.800744644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:25:42.530845253 +0000 UTC m=+807.177591945" watchObservedRunningTime="2026-06-09 14:25:42.531809501 +0000 UTC m=+807.178556194" Jun 09 14:25:48.499686 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:48.499660 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6c8bd7b74c-nwhpz" Jun 09 14:25:53.496338 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:53.496303 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:25:53.496907 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:25:53.496382 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-p9wg6" Jun 09 14:26:28.876748 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:28.876700 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:28.943866 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:28.943830 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:33.005014 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:33.004981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:36.869630 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:36.869594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:36.926700 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:36.926672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:54.515116 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.515075 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg"] Jun 09 14:26:54.518490 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.518469 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.520784 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.520747 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 14:26:54.520918 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.520809 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vn2nc\"" Jun 09 14:26:54.521751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.521734 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 14:26:54.521835 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.521765 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 09 14:26:54.529659 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.529631 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg"] Jun 09 14:26:54.543991 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.543899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:54.619345 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.619314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4cxc\" (UniqueName: \"kubernetes.io/projected/df01006a-c64f-4392-91c7-a20cb81da3af-kube-api-access-q4cxc\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.619524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.619355 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.619524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.619388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.619524 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.619477 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.619634 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.619552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.619634 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.619588 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df01006a-c64f-4392-91c7-a20cb81da3af-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.720882 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.720838 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721076 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.720888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df01006a-c64f-4392-91c7-a20cb81da3af-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721076 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.720932 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4cxc\" (UniqueName: \"kubernetes.io/projected/df01006a-c64f-4392-91c7-a20cb81da3af-kube-api-access-q4cxc\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721076 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.720952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721076 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.720993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721076 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.721019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721351 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.721304 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721443 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.721373 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.721501 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.721449 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.723375 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.723355 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df01006a-c64f-4392-91c7-a20cb81da3af-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.723703 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.723685 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df01006a-c64f-4392-91c7-a20cb81da3af-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.728431 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.728376 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4cxc\" (UniqueName: \"kubernetes.io/projected/df01006a-c64f-4392-91c7-a20cb81da3af-kube-api-access-q4cxc\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg\" (UID: \"df01006a-c64f-4392-91c7-a20cb81da3af\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.853861 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.853759 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:26:54.997680 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:54.997652 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg"] Jun 09 14:26:54.999520 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:26:54.999494 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf01006a_c64f_4392_91c7_a20cb81da3af.slice/crio-227294846ca07149b835f33bb0f975fbf7e05243225916175494bbb8b600e527 WatchSource:0}: Error finding container 227294846ca07149b835f33bb0f975fbf7e05243225916175494bbb8b600e527: Status 404 returned error can't find the container with id 227294846ca07149b835f33bb0f975fbf7e05243225916175494bbb8b600e527 Jun 09 14:26:55.488230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:55.488196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:26:55.895709 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:55.895588 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" event={"ID":"df01006a-c64f-4392-91c7-a20cb81da3af","Type":"ContainerStarted","Data":"227294846ca07149b835f33bb0f975fbf7e05243225916175494bbb8b600e527"} Jun 09 14:26:59.922723 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:26:59.922686 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" event={"ID":"df01006a-c64f-4392-91c7-a20cb81da3af","Type":"ContainerStarted","Data":"c502ed188232b3aed5426e8a70421e87eec23285090f00ea976cb855e0c15fb2"} Jun 09 14:27:00.929348 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:00.929310 2568 generic.go:356] "Generic (PLEG): container finished" podID="df01006a-c64f-4392-91c7-a20cb81da3af" containerID="c502ed188232b3aed5426e8a70421e87eec23285090f00ea976cb855e0c15fb2" exitCode=0 Jun 09 14:27:00.929781 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:00.929417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" event={"ID":"df01006a-c64f-4392-91c7-a20cb81da3af","Type":"ContainerDied","Data":"c502ed188232b3aed5426e8a70421e87eec23285090f00ea976cb855e0c15fb2"} Jun 09 14:27:02.943344 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:02.943310 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" event={"ID":"df01006a-c64f-4392-91c7-a20cb81da3af","Type":"ContainerStarted","Data":"f66ddbda0fa96e9333ccd0b9ef377854bb0833af8c6f5351bdd8fe0ac52d37d5"} Jun 09 14:27:02.962602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:02.962542 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" podStartSLOduration=2.066121699 podStartE2EDuration="8.962523659s" totalImagesPullingTime="5.51029767s" totalInitContainerRuntime="1.38610429s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:26:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:27:02.961651956 +0000 UTC m=+887.608398649" watchObservedRunningTime="2026-06-09 14:27:02.962523659 +0000 UTC m=+887.609270351" Jun 09 14:27:04.854272 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:04.854230 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:27:04.854779 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:04.854305 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:27:04.868030 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:04.867999 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:27:04.966027 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:04.965992 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg" Jun 09 14:27:07.315180 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:07.315142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:27:08.381200 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.381166 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:27:08.722232 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.722147 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg"] Jun 09 14:27:08.732833 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.732798 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.738258 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.738220 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 09 14:27:08.740273 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.740244 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg"] Jun 09 14:27:08.756709 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.756680 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:27:08.864605 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.864570 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.864605 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.864610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.864894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.864640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7ssp\" (UniqueName: \"kubernetes.io/projected/5d8d5291-ef79-459a-91b8-cb40d533c29d-kube-api-access-x7ssp\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.864894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.864739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.864894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.864772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.864894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.864806 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d8d5291-ef79-459a-91b8-cb40d533c29d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.965727 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.965681 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.965930 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.965735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.965930 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.965773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7ssp\" (UniqueName: \"kubernetes.io/projected/5d8d5291-ef79-459a-91b8-cb40d533c29d-kube-api-access-x7ssp\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.965930 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.965816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.965930 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.965854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.965930 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.965894 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d8d5291-ef79-459a-91b8-cb40d533c29d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.966214 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.966150 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.966309 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.966286 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.966356 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.966333 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.968618 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.968586 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5d8d5291-ef79-459a-91b8-cb40d533c29d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.968860 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.968830 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d8d5291-ef79-459a-91b8-cb40d533c29d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:08.973498 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:08.973432 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7ssp\" (UniqueName: \"kubernetes.io/projected/5d8d5291-ef79-459a-91b8-cb40d533c29d-kube-api-access-x7ssp\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-mg9qg\" (UID: \"5d8d5291-ef79-459a-91b8-cb40d533c29d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:09.048858 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:09.048764 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:27:09.202182 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:09.202155 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg"] Jun 09 14:27:09.204696 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:27:09.204664 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d8d5291_ef79_459a_91b8_cb40d533c29d.slice/crio-d75d9bd94ae542651a5c66821e6c4826e85990a24c39a46a636960db168fd74a WatchSource:0}: Error finding container d75d9bd94ae542651a5c66821e6c4826e85990a24c39a46a636960db168fd74a: Status 404 returned error can't find the container with id d75d9bd94ae542651a5c66821e6c4826e85990a24c39a46a636960db168fd74a Jun 09 14:27:09.982770 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:09.982733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" event={"ID":"5d8d5291-ef79-459a-91b8-cb40d533c29d","Type":"ContainerStarted","Data":"9522a323a52a89dd7636f7b950ea26b66f839a6341deac661f51db65c86d5666"} Jun 09 14:27:09.983180 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:09.982791 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" event={"ID":"5d8d5291-ef79-459a-91b8-cb40d533c29d","Type":"ContainerStarted","Data":"d75d9bd94ae542651a5c66821e6c4826e85990a24c39a46a636960db168fd74a"} Jun 09 14:27:12.240634 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:12.240598 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:27:15.883129 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:15.883098 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:27:15.884609 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:15.884586 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:27:20.112620 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:20.112588 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:27:42.155598 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:42.155559 2568 generic.go:356] "Generic (PLEG): container finished" podID="ff0c800c-89d1-42ab-9b21-91da52b54347" containerID="680dae60a20602c16cb512e7df44d119d3141e8fe64db4a9d2a5ee175674d935" exitCode=1 Jun 09 14:27:42.156034 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:42.155635 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerDied","Data":"680dae60a20602c16cb512e7df44d119d3141e8fe64db4a9d2a5ee175674d935"} Jun 09 14:27:42.156034 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:42.155985 2568 scope.go:117] "RemoveContainer" containerID="680dae60a20602c16cb512e7df44d119d3141e8fe64db4a9d2a5ee175674d935" Jun 09 14:27:43.164549 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:43.164510 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerStarted","Data":"9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d"} Jun 09 14:27:43.165035 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:43.164726 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:27:43.531612 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:43.531571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:27:54.173395 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:27:54.173365 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:28:03.278245 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:03.278212 2568 generic.go:356] "Generic (PLEG): container finished" podID="5d8d5291-ef79-459a-91b8-cb40d533c29d" containerID="9522a323a52a89dd7636f7b950ea26b66f839a6341deac661f51db65c86d5666" exitCode=0 Jun 09 14:28:03.278693 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:03.278286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" event={"ID":"5d8d5291-ef79-459a-91b8-cb40d533c29d","Type":"ContainerDied","Data":"9522a323a52a89dd7636f7b950ea26b66f839a6341deac661f51db65c86d5666"} Jun 09 14:28:04.284746 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:04.284712 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" event={"ID":"5d8d5291-ef79-459a-91b8-cb40d533c29d","Type":"ContainerStarted","Data":"48015a11598b3289e0c513b3467315392fa0d8e50a393cc3b42dd97e9ceaf774"} Jun 09 14:28:04.302894 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:04.302829 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" podStartSLOduration=2.596475931 podStartE2EDuration="56.302809385s" totalImagesPullingTime="254.507104ms" totalInitContainerRuntime="53.45182635s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:28:04.301750859 +0000 UTC m=+948.948497543" watchObservedRunningTime="2026-06-09 14:28:04.302809385 +0000 UTC m=+948.949556078" Jun 09 14:28:05.711251 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.711027 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr"] Jun 09 14:28:05.713949 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.713928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.718487 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.718466 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 09 14:28:05.727936 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.727914 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr"] Jun 09 14:28:05.751368 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.751330 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:28:05.805780 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.805741 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.805969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.805799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.805969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.805850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.805969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.805873 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2281623d-2a41-4312-8389-a5f279018873-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.805969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.805929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgctq\" (UniqueName: \"kubernetes.io/projected/2281623d-2a41-4312-8389-a5f279018873-kube-api-access-kgctq\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.806139 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.805980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.906552 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.906509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgctq\" (UniqueName: \"kubernetes.io/projected/2281623d-2a41-4312-8389-a5f279018873-kube-api-access-kgctq\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.906751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.906570 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.906751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.906632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.906751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.906669 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.906751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.906712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.906751 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.906735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2281623d-2a41-4312-8389-a5f279018873-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.907174 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.907137 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.907302 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.907179 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.907302 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.907195 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.909151 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.909126 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2281623d-2a41-4312-8389-a5f279018873-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.909444 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.909396 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2281623d-2a41-4312-8389-a5f279018873-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:05.913902 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:05.913878 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgctq\" (UniqueName: \"kubernetes.io/projected/2281623d-2a41-4312-8389-a5f279018873-kube-api-access-kgctq\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-r65jr\" (UID: \"2281623d-2a41-4312-8389-a5f279018873\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:06.027336 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:06.027295 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:28:06.173131 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:06.173106 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr"] Jun 09 14:28:06.174885 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:28:06.174858 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2281623d_2a41_4312_8389_a5f279018873.slice/crio-10fb67e943c7dcbd5826a254e25834fce444bad12585b964cdab2566ed44009f WatchSource:0}: Error finding container 10fb67e943c7dcbd5826a254e25834fce444bad12585b964cdab2566ed44009f: Status 404 returned error can't find the container with id 10fb67e943c7dcbd5826a254e25834fce444bad12585b964cdab2566ed44009f Jun 09 14:28:06.296394 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:06.296305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" event={"ID":"2281623d-2a41-4312-8389-a5f279018873","Type":"ContainerStarted","Data":"5cc4779467ffe73ea53064f9da7d99548b2e8806b24ca4a61e01b2f35ba4f3ff"} Jun 09 14:28:06.296394 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:06.296350 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" event={"ID":"2281623d-2a41-4312-8389-a5f279018873","Type":"ContainerStarted","Data":"10fb67e943c7dcbd5826a254e25834fce444bad12585b964cdab2566ed44009f"} Jun 09 14:28:09.049412 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:09.049371 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:28:09.049867 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:09.049488 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:28:09.063085 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:09.063058 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:28:09.323636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:09.323559 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-mg9qg" Jun 09 14:28:59.709010 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:28:59.708973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:27.333098 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:27.333065 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:27.381951 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:27.381919 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:27.746007 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:27.745975 2568 generic.go:356] "Generic (PLEG): container finished" podID="2281623d-2a41-4312-8389-a5f279018873" containerID="5cc4779467ffe73ea53064f9da7d99548b2e8806b24ca4a61e01b2f35ba4f3ff" exitCode=0 Jun 09 14:29:27.746163 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:27.746052 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" event={"ID":"2281623d-2a41-4312-8389-a5f279018873","Type":"ContainerDied","Data":"5cc4779467ffe73ea53064f9da7d99548b2e8806b24ca4a61e01b2f35ba4f3ff"} Jun 09 14:29:28.752968 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:28.752936 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" event={"ID":"2281623d-2a41-4312-8389-a5f279018873","Type":"ContainerStarted","Data":"a84505c3ad93c7e364e5c0c8ee7d4bb763438826727e7e66308de94c7796bcb6"} Jun 09 14:29:28.771567 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:28.771525 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" podStartSLOduration=2.184808512 podStartE2EDuration="1m23.771511627s" totalImagesPullingTime="322.545472ms" totalInitContainerRuntime="1m21.264157643s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:29:28.768702576 +0000 UTC m=+1033.415449270" watchObservedRunningTime="2026-06-09 14:29:28.771511627 +0000 UTC m=+1033.418258319" Jun 09 14:29:36.027991 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:36.027955 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:29:36.028500 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:36.028006 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:29:36.040570 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:36.040542 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:29:36.808192 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:36.808159 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-r65jr" Jun 09 14:29:40.532636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:40.532599 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:40.631545 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:40.631506 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:41.625043 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:41.625004 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:41.727830 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:41.727801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:29:42.831717 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:42.831687 2568 generic.go:356] "Generic (PLEG): container finished" podID="ff0c800c-89d1-42ab-9b21-91da52b54347" containerID="9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d" exitCode=1 Jun 09 14:29:42.832120 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:42.831771 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerDied","Data":"9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d"} Jun 09 14:29:42.832120 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:42.831835 2568 scope.go:117] "RemoveContainer" containerID="680dae60a20602c16cb512e7df44d119d3141e8fe64db4a9d2a5ee175674d935" Jun 09 14:29:42.832230 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:42.832140 2568 scope.go:117] "RemoveContainer" containerID="9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d" Jun 09 14:29:42.832384 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:29:42.832361 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:29:44.171693 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:44.171660 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:29:44.172173 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:44.172056 2568 scope.go:117] "RemoveContainer" containerID="9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d" Jun 09 14:29:44.172307 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:29:44.172282 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:29:56.871515 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:56.871442 2568 scope.go:117] "RemoveContainer" containerID="9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d" Jun 09 14:29:57.925793 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:57.925756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerStarted","Data":"572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9"} Jun 09 14:29:57.926164 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:29:57.925985 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:30:08.933498 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:30:08.933462 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:31:23.009735 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:23.009700 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:31:51.363663 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:51.363576 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:31:51.412594 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:51.412559 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:31:57.572056 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:57.572023 2568 generic.go:356] "Generic (PLEG): container finished" podID="ff0c800c-89d1-42ab-9b21-91da52b54347" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" exitCode=1 Jun 09 14:31:57.572480 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:57.572107 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerDied","Data":"572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9"} Jun 09 14:31:57.572480 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:57.572181 2568 scope.go:117] "RemoveContainer" containerID="9faca71494fd9f043e54fc588a6c3fd1f62e19b9c45ccc75a234bf436f21176d" Jun 09 14:31:57.572598 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:57.572526 2568 scope.go:117] "RemoveContainer" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" Jun 09 14:31:57.572803 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:31:57.572780 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:31:58.931679 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:58.931646 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:31:58.932125 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:58.931984 2568 scope.go:117] "RemoveContainer" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" Jun 09 14:31:58.932198 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:31:58.932180 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:31:59.448028 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:59.447989 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:31:59.586855 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:31:59.586822 2568 scope.go:117] "RemoveContainer" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" Jun 09 14:31:59.587117 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:31:59.587091 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:32:05.934621 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:05.934590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:32:06.049788 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.049747 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-t5fdz"] Jun 09 14:32:06.049986 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.049944 2568 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" podUID="c12bfd07-e56f-4840-958e-b595012b4acd" containerName="manager" containerID="cri-o://bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91" gracePeriod=30 Jun 09 14:32:06.321602 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.321579 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:32:06.423336 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.423303 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert\") pod \"c12bfd07-e56f-4840-958e-b595012b4acd\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " Jun 09 14:32:06.423590 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.423386 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbvp8\" (UniqueName: \"kubernetes.io/projected/c12bfd07-e56f-4840-958e-b595012b4acd-kube-api-access-xbvp8\") pod \"c12bfd07-e56f-4840-958e-b595012b4acd\" (UID: \"c12bfd07-e56f-4840-958e-b595012b4acd\") " Jun 09 14:32:06.425586 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.425550 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c12bfd07-e56f-4840-958e-b595012b4acd" (UID: "c12bfd07-e56f-4840-958e-b595012b4acd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 14:32:06.425685 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.425643 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12bfd07-e56f-4840-958e-b595012b4acd-kube-api-access-xbvp8" (OuterVolumeSpecName: "kube-api-access-xbvp8") pod "c12bfd07-e56f-4840-958e-b595012b4acd" (UID: "c12bfd07-e56f-4840-958e-b595012b4acd"). InnerVolumeSpecName "kube-api-access-xbvp8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 14:32:06.524128 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.524095 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbvp8\" (UniqueName: \"kubernetes.io/projected/c12bfd07-e56f-4840-958e-b595012b4acd-kube-api-access-xbvp8\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:32:06.524128 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.524124 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c12bfd07-e56f-4840-958e-b595012b4acd-webhook-cert\") on node \"ip-10-0-128-214.ec2.internal\" DevicePath \"\"" Jun 09 14:32:06.626177 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.626140 2568 generic.go:356] "Generic (PLEG): container finished" podID="c12bfd07-e56f-4840-958e-b595012b4acd" containerID="bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91" exitCode=0 Jun 09 14:32:06.626341 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.626183 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" Jun 09 14:32:06.626341 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.626192 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" event={"ID":"c12bfd07-e56f-4840-958e-b595012b4acd","Type":"ContainerDied","Data":"bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91"} Jun 09 14:32:06.626341 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.626241 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585c5cbb6c-t5fdz" event={"ID":"c12bfd07-e56f-4840-958e-b595012b4acd","Type":"ContainerDied","Data":"a23417e109a290e5bda2f1a9a1b7ca6ce4b5d48ed688cb0dc71df81516e5fdfe"} Jun 09 14:32:06.626341 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.626265 2568 scope.go:117] "RemoveContainer" containerID="bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91" Jun 09 14:32:06.635617 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.635600 2568 scope.go:117] "RemoveContainer" containerID="bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91" Jun 09 14:32:06.635862 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:32:06.635843 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91\": container with ID starting with bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91 not found: ID does not exist" containerID="bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91" Jun 09 14:32:06.635904 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.635871 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91"} err="failed to get container status \"bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91\": rpc error: code = NotFound desc = could not find container \"bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91\": container with ID starting with bf2b91734cc3a8a6f42183dd9997383556482083f41f6373ab1bf02fb12bef91 not found: ID does not exist" Jun 09 14:32:06.647638 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.647600 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-t5fdz"] Jun 09 14:32:06.651609 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:06.651582 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-t5fdz"] Jun 09 14:32:07.303636 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.303594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:32:07.414587 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.414548 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-txb8t"] Jun 09 14:32:07.415002 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.414989 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12bfd07-e56f-4840-958e-b595012b4acd" containerName="manager" Jun 09 14:32:07.415050 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.415005 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12bfd07-e56f-4840-958e-b595012b4acd" containerName="manager" Jun 09 14:32:07.415125 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.415107 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12bfd07-e56f-4840-958e-b595012b4acd" containerName="manager" Jun 09 14:32:07.419467 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.419446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.422814 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.422793 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 14:32:07.422948 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.422792 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mljdg\"" Jun 09 14:32:07.422948 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.422792 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 14:32:07.428561 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.428535 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-txb8t"] Jun 09 14:32:07.431502 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.431480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cblr5\" (UniqueName: \"kubernetes.io/projected/c634b229-f1eb-4a6b-97c7-83035be69d4a-kube-api-access-cblr5\") pod \"maas-controller-585c5cbb6c-txb8t\" (UID: \"c634b229-f1eb-4a6b-97c7-83035be69d4a\") " pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.431626 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.431520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c634b229-f1eb-4a6b-97c7-83035be69d4a-webhook-cert\") pod \"maas-controller-585c5cbb6c-txb8t\" (UID: \"c634b229-f1eb-4a6b-97c7-83035be69d4a\") " pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.532863 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.532811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cblr5\" (UniqueName: \"kubernetes.io/projected/c634b229-f1eb-4a6b-97c7-83035be69d4a-kube-api-access-cblr5\") pod \"maas-controller-585c5cbb6c-txb8t\" (UID: \"c634b229-f1eb-4a6b-97c7-83035be69d4a\") " pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.533062 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.532889 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c634b229-f1eb-4a6b-97c7-83035be69d4a-webhook-cert\") pod \"maas-controller-585c5cbb6c-txb8t\" (UID: \"c634b229-f1eb-4a6b-97c7-83035be69d4a\") " pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.535526 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.535497 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c634b229-f1eb-4a6b-97c7-83035be69d4a-webhook-cert\") pod \"maas-controller-585c5cbb6c-txb8t\" (UID: \"c634b229-f1eb-4a6b-97c7-83035be69d4a\") " pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.541354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.541332 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cblr5\" (UniqueName: \"kubernetes.io/projected/c634b229-f1eb-4a6b-97c7-83035be69d4a-kube-api-access-cblr5\") pod \"maas-controller-585c5cbb6c-txb8t\" (UID: \"c634b229-f1eb-4a6b-97c7-83035be69d4a\") " pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.730597 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.730562 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:07.867899 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.867738 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-585c5cbb6c-txb8t"] Jun 09 14:32:07.870290 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:32:07.870262 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc634b229_f1eb_4a6b_97c7_83035be69d4a.slice/crio-20ec9aa589846a5f76de3e394c016781fc61518233833c918f24bd214d164a00 WatchSource:0}: Error finding container 20ec9aa589846a5f76de3e394c016781fc61518233833c918f24bd214d164a00: Status 404 returned error can't find the container with id 20ec9aa589846a5f76de3e394c016781fc61518233833c918f24bd214d164a00 Jun 09 14:32:07.871734 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.871716 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 14:32:07.878883 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:07.878856 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c12bfd07-e56f-4840-958e-b595012b4acd" path="/var/lib/kubelet/pods/c12bfd07-e56f-4840-958e-b595012b4acd/volumes" Jun 09 14:32:08.643098 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:08.642997 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" event={"ID":"c634b229-f1eb-4a6b-97c7-83035be69d4a","Type":"ContainerStarted","Data":"bbee3c2e497f8a2e69c20eeb3f00549ba8673a5dd4c08f8f5afbc6c8cd416968"} Jun 09 14:32:08.643098 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:08.643058 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" event={"ID":"c634b229-f1eb-4a6b-97c7-83035be69d4a","Type":"ContainerStarted","Data":"20ec9aa589846a5f76de3e394c016781fc61518233833c918f24bd214d164a00"} Jun 09 14:32:08.643098 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:08.643085 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:08.661136 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:08.661080 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" podStartSLOduration=1.191170739 podStartE2EDuration="1.661063712s" totalImagesPullingTime="469.892973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 14:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:32:08.658874322 +0000 UTC m=+1193.305621007" watchObservedRunningTime="2026-06-09 14:32:08.661063712 +0000 UTC m=+1193.307810450" Jun 09 14:32:10.871322 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:10.871290 2568 scope.go:117] "RemoveContainer" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" Jun 09 14:32:10.871797 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:32:10.871508 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:32:15.914985 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:15.914955 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:32:15.921427 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:15.921388 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2wmnf_8393f4ed-5c8d-4685-8bc0-87d143940155/ovn-acl-logging/0.log" Jun 09 14:32:19.654994 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:19.654966 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-585c5cbb6c-txb8t" Jun 09 14:32:23.871904 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:23.871874 2568 scope.go:117] "RemoveContainer" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" Jun 09 14:32:24.732354 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:24.732322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerStarted","Data":"2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f"} Jun 09 14:32:24.732591 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:24.732571 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:32:35.739167 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:35.739135 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:32:51.518549 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:51.518501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:32:52.596683 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:32:52.596649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:33:00.041635 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:33:00.041559 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:33:05.786981 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:33:05.786944 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:33:05.853856 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:33:05.853829 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:33:37.890813 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:33:37.890763 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:33:37.984267 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:33:37.984229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:01.871875 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:01.871836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:07.647130 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:07.647081 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:07.710428 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:07.710351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:07.782615 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:07.782575 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:07.882642 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:07.882551 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:08.015718 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:08.015685 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:08.121776 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:08.121741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:24.381452 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.381337 2568 generic.go:356] "Generic (PLEG): container finished" podID="ff0c800c-89d1-42ab-9b21-91da52b54347" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" exitCode=1 Jun 09 14:34:24.381452 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.381427 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerDied","Data":"2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f"} Jun 09 14:34:24.382007 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.381493 2568 scope.go:117] "RemoveContainer" containerID="572a9043d4e3daad7df03ae9bc55e0a40200af56a515ea450fd808dff0fedfb9" Jun 09 14:34:24.382007 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.381843 2568 scope.go:117] "RemoveContainer" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" Jun 09 14:34:24.382217 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:34:24.382101 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:34:24.651905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.651823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:24.774699 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.774667 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:24.890557 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:24.890498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:25.737900 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:25.737805 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:34:25.738355 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:25.738194 2568 scope.go:117] "RemoveContainer" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" Jun 09 14:34:25.738530 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:34:25.738506 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:34:30.553876 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:30.553812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:30.610978 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:30.610919 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:38.710046 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:38.710010 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:38.834771 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:38.834737 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:38.944122 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:38.944091 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:39.044373 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:39.044343 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:39.448002 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:39.447920 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:34:39.448366 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:39.448351 2568 scope.go:117] "RemoveContainer" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" Jun 09 14:34:39.448606 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:34:39.448583 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:34:51.871933 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:51.871895 2568 scope.go:117] "RemoveContainer" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" Jun 09 14:34:51.872342 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:34:51.872248 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:34:55.488566 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:55.488536 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:55.598854 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:55.598824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:34:55.707380 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:34:55.707347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:35:01.400832 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:01.400788 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:35:01.470032 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:01.469998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:35:02.871495 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:02.871466 2568 scope.go:117] "RemoveContainer" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" Jun 09 14:35:02.871869 ip-10-0-128-214 kubenswrapper[2568]: E0609 14:35:02.871670 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-2rj9c_openshift-ingress(ff0c800c-89d1-42ab-9b21-91da52b54347)\"" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" podUID="ff0c800c-89d1-42ab-9b21-91da52b54347" Jun 09 14:35:14.800988 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:14.800957 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-986494677-9pr9d_d3e182c6-611b-415e-bdc6-1cc1bd1b9d45/nginx/0.log" Jun 09 14:35:14.911813 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:14.911784 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-bnb54_56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0/manager/0.log" Jun 09 14:35:15.035358 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.035333 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-zppv6_44185177-68e6-4347-91e0-f4bf207f810a/manager/0.log" Jun 09 14:35:15.152652 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.152578 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c8bd7b74c-nwhpz_88138061-245f-49f5-ae80-3007e57a8eb3/maas-api/0.log" Jun 09 14:35:15.383948 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.383917 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-585c5cbb6c-txb8t_c634b229-f1eb-4a6b-97c7-83035be69d4a/manager/0.log" Jun 09 14:35:15.502529 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.502503 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-zltjm_2fe01e53-7e7f-4e81-ad29-ca40fa3af222/server/0.log" Jun 09 14:35:15.617551 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.617526 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-l8fbt_31edc12e-1f61-48f2-b550-628cd0e6b021/manager/0.log" Jun 09 14:35:15.732308 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.732282 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c57c599f5-26gnw_3fb57b60-574a-4454-8cf3-d6505fc762a3/manager/0.log" Jun 09 14:35:15.737941 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.737920 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c57c599f5-26gnw_3fb57b60-574a-4454-8cf3-d6505fc762a3/copy-manifests/0.log" Jun 09 14:35:15.874206 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:15.874124 2568 scope.go:117] "RemoveContainer" containerID="2329202c9bc6efa92549444f685ae91b4495d2e003eb53ba255789495373027f" Jun 09 14:35:16.679696 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:16.679660 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" event={"ID":"ff0c800c-89d1-42ab-9b21-91da52b54347","Type":"ContainerStarted","Data":"86cd6e6006ce84bef48a0caa81b1df41b25a0fbec7938305708741cc8ce6a80b"} Jun 09 14:35:16.679893 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:16.679878 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:35:17.509076 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:17.509047 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nr2gb_b3011c70-5142-44ea-b257-fea30b142bee/manager/0.log" Jun 09 14:35:17.973974 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:17.973888 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-56hq7_c6119b4f-8267-44a9-a8d4-b3c22544f495/manager/0.log" Jun 09 14:35:18.686346 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:18.686318 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-2tn87_e08c5d6f-2a71-4d36-96cb-76aa5d5205b4/discovery/0.log" Jun 09 14:35:18.908160 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:18.908129 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df8999ffb-wff2z_fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c/kube-auth-proxy/0.log" Jun 09 14:35:19.143915 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:19.143876 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p9wg6_a5730f4a-2886-409d-b625-2e92ee477f88/payload-pre-processing/0.log" Jun 09 14:35:19.248678 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:19.248653 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/4.log" Jun 09 14:35:19.732616 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:19.732588 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg_df01006a-c64f-4392-91c7-a20cb81da3af/storage-initializer/0.log" Jun 09 14:35:19.740236 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:19.740219 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-ngmlg_df01006a-c64f-4392-91c7-a20cb81da3af/main/0.log" Jun 09 14:35:19.854918 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:19.854891 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-mg9qg_5d8d5291-ef79-459a-91b8-cb40d533c29d/storage-initializer/0.log" Jun 09 14:35:19.862199 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:19.862179 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-mg9qg_5d8d5291-ef79-459a-91b8-cb40d533c29d/main/0.log" Jun 09 14:35:20.344478 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:20.344453 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-r65jr_2281623d-2a41-4312-8389-a5f279018873/storage-initializer/0.log" Jun 09 14:35:20.351346 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:20.351323 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-r65jr_2281623d-2a41-4312-8389-a5f279018873/main/0.log" Jun 09 14:35:24.932993 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:24.932955 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:35:25.033115 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:25.033083 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" limit=1024 Jun 09 14:35:27.687569 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:27.687540 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-2rj9c" Jun 09 14:35:27.944924 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:27.944840 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bcb8m_4a611bc1-9997-42fa-88ae-2feb6105e7bd/global-pull-secret-syncer/0.log" Jun 09 14:35:28.067905 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:28.067878 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7nrcw_e5215ace-4db9-42de-ac5d-a0a68738cbbb/konnectivity-agent/0.log" Jun 09 14:35:28.132293 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:28.132265 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-214.ec2.internal_a72335425bac515a27766cdcfbe29358/haproxy/0.log" Jun 09 14:35:32.205343 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:32.205299 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-cr5br_64e43e93-2e5d-4c1c-bdb1-60bbdea15eb7/sail-operator/0.log" Jun 09 14:35:33.244113 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:33.240705 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-nr2gb_b3011c70-5142-44ea-b257-fea30b142bee/manager/0.log" Jun 09 14:35:33.361942 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:33.361898 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-56hq7_c6119b4f-8267-44a9-a8d4-b3c22544f495/manager/0.log" Jun 09 14:35:35.403748 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.403717 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-52gfq_ef8f335f-ed20-433c-85ad-b8199bee7c60/cluster-monitoring-operator/0.log" Jun 09 14:35:35.443442 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.443369 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-btdtp_01920cf4-dcdc-4e0e-81be-bacc6278e1e8/kube-state-metrics/0.log" Jun 09 14:35:35.461632 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.461608 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-btdtp_01920cf4-dcdc-4e0e-81be-bacc6278e1e8/kube-rbac-proxy-main/0.log" Jun 09 14:35:35.481208 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.481178 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-btdtp_01920cf4-dcdc-4e0e-81be-bacc6278e1e8/kube-rbac-proxy-self/0.log" Jun 09 14:35:35.540863 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.540837 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-mw5w6_3dc5e069-b16e-44a8-bc5e-840ac232447f/monitoring-plugin/0.log" Jun 09 14:35:35.721991 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.721923 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t2w69_04d51df8-e3ec-4d08-88a6-6899e78091d2/node-exporter/0.log" Jun 09 14:35:35.741564 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.741543 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t2w69_04d51df8-e3ec-4d08-88a6-6899e78091d2/kube-rbac-proxy/0.log" Jun 09 14:35:35.760433 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.760387 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t2w69_04d51df8-e3ec-4d08-88a6-6899e78091d2/init-textfile/0.log" Jun 09 14:35:35.787149 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.787118 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dpxx2_a7b4fad4-a36f-4027-bb1e-8dd738bf390f/kube-rbac-proxy-main/0.log" Jun 09 14:35:35.812007 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.811985 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dpxx2_a7b4fad4-a36f-4027-bb1e-8dd738bf390f/kube-rbac-proxy-self/0.log" Jun 09 14:35:35.834014 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:35.833988 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dpxx2_a7b4fad4-a36f-4027-bb1e-8dd738bf390f/openshift-state-metrics/0.log" Jun 09 14:35:36.071843 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.071816 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-btpq8_89191b49-2d6c-45a0-a2b0-abc00566415c/prometheus-operator-admission-webhook/0.log" Jun 09 14:35:36.175870 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.175843 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5584564fcf-g6jt7_48320bc1-3572-425f-aaf0-7b389a9a1568/thanos-query/0.log" Jun 09 14:35:36.196446 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.196396 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5584564fcf-g6jt7_48320bc1-3572-425f-aaf0-7b389a9a1568/kube-rbac-proxy-web/0.log" Jun 09 14:35:36.214758 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.214736 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5584564fcf-g6jt7_48320bc1-3572-425f-aaf0-7b389a9a1568/kube-rbac-proxy/0.log" Jun 09 14:35:36.233275 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.233253 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5584564fcf-g6jt7_48320bc1-3572-425f-aaf0-7b389a9a1568/prom-label-proxy/0.log" Jun 09 14:35:36.252871 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.252836 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5584564fcf-g6jt7_48320bc1-3572-425f-aaf0-7b389a9a1568/kube-rbac-proxy-rules/0.log" Jun 09 14:35:36.272484 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:36.272466 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5584564fcf-g6jt7_48320bc1-3572-425f-aaf0-7b389a9a1568/kube-rbac-proxy-metrics/0.log" Jun 09 14:35:37.480722 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:37.480692 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-6g4j5_8ebca47e-d091-429d-a238-2583e0da2bd3/networking-console-plugin/0.log" Jun 09 14:35:38.550578 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.550543 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj"] Jun 09 14:35:38.553974 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.553954 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.556566 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.556542 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mncq\"/\"kube-root-ca.crt\"" Jun 09 14:35:38.557040 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.557015 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mncq\"/\"openshift-service-ca.crt\"" Jun 09 14:35:38.557143 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.557039 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8mncq\"/\"default-dockercfg-l4ftw\"" Jun 09 14:35:38.563570 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.563542 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj"] Jun 09 14:35:38.598388 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.598366 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-779d6b5c65-wxjgq_26df8fe8-46e8-4f88-be5f-9732eb69b77a/console/0.log" Jun 09 14:35:38.625436 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.625414 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-8swkf_eeca0437-0985-4c2b-907b-08d00f3fc0bd/download-server/0.log" Jun 09 14:35:38.690092 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.690057 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-proc\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.690213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.690099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm5db\" (UniqueName: \"kubernetes.io/projected/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-kube-api-access-cm5db\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.690213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.690152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-lib-modules\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.690213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.690177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-podres\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.690213 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.690196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-sys\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791378 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791340 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-proc\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791553 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791384 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm5db\" (UniqueName: \"kubernetes.io/projected/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-kube-api-access-cm5db\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791553 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-lib-modules\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791553 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791473 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-proc\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791564 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-podres\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791573 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-lib-modules\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791661 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791602 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-sys\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791761 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791678 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-sys\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.791761 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.791689 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-podres\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.798835 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.798808 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm5db\" (UniqueName: \"kubernetes.io/projected/6ef6f790-40cc-4d80-95a0-1d2ee21fc314-kube-api-access-cm5db\") pod \"perf-node-gather-daemonset-d27hj\" (UID: \"6ef6f790-40cc-4d80-95a0-1d2ee21fc314\") " pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:38.869840 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:38.869772 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:39.005690 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:39.005668 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj"] Jun 09 14:35:39.008355 ip-10-0-128-214 kubenswrapper[2568]: W0609 14:35:39.008325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6ef6f790_40cc_4d80_95a0_1d2ee21fc314.slice/crio-b85f5f8ff313e850395491104f88df725872159e1a9d8ee4a614ed157d4eba0b WatchSource:0}: Error finding container b85f5f8ff313e850395491104f88df725872159e1a9d8ee4a614ed157d4eba0b: Status 404 returned error can't find the container with id b85f5f8ff313e850395491104f88df725872159e1a9d8ee4a614ed157d4eba0b Jun 09 14:35:39.328250 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:39.328221 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-8xkfr_02143712-146d-41ab-9b7d-b1da35dce3bf/volume-data-source-validator/0.log" Jun 09 14:35:39.807182 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:39.807146 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" event={"ID":"6ef6f790-40cc-4d80-95a0-1d2ee21fc314","Type":"ContainerStarted","Data":"b532f138e026842f09c64320c6fd8b3437f743a83818e6a9c8b5abc0998b928c"} Jun 09 14:35:39.807647 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:39.807204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" event={"ID":"6ef6f790-40cc-4d80-95a0-1d2ee21fc314","Type":"ContainerStarted","Data":"b85f5f8ff313e850395491104f88df725872159e1a9d8ee4a614ed157d4eba0b"} Jun 09 14:35:39.807647 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:39.807349 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:39.822804 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:39.822760 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" podStartSLOduration=1.822748382 podStartE2EDuration="1.822748382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 14:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 14:35:39.821143662 +0000 UTC m=+1404.467890376" watchObservedRunningTime="2026-06-09 14:35:39.822748382 +0000 UTC m=+1404.469495126" Jun 09 14:35:40.445699 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:40.445667 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-54mcq_a06ae45d-d42d-4ed3-9770-95289acac343/dns/0.log" Jun 09 14:35:40.466542 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:40.466512 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-54mcq_a06ae45d-d42d-4ed3-9770-95289acac343/kube-rbac-proxy/0.log" Jun 09 14:35:40.619994 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:40.619965 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rk89k_8da21813-284f-4169-b06c-dd9b636cdc8f/dns-node-resolver/0.log" Jun 09 14:35:41.137265 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:41.137233 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5499bf57fb-7fq4w_f61f8351-2bf3-4012-82ea-77ee5fa864a0/registry/0.log" Jun 09 14:35:41.157701 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:41.157678 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vvsn4_a18c19c3-e11d-4191-81a4-73707986bee7/node-ca/0.log" Jun 09 14:35:42.170720 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:42.170681 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-2tn87_e08c5d6f-2a71-4d36-96cb-76aa5d5205b4/discovery/0.log" Jun 09 14:35:42.212172 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:42.212144 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df8999ffb-wff2z_fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c/kube-auth-proxy/0.log" Jun 09 14:35:42.298562 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:42.298528 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p9wg6_a5730f4a-2886-409d-b625-2e92ee477f88/payload-pre-processing/0.log" Jun 09 14:35:42.329757 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:42.329706 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/4.log" Jun 09 14:35:42.331906 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:42.331882 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/3.log" Jun 09 14:35:42.938113 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:42.938085 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hvcpz_d6221db5-4346-494c-85c3-9cf89b8cba98/serve-healthcheck-canary/0.log" Jun 09 14:35:43.479510 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:43.479461 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-hf2bv_023aaede-6273-4afe-8d49-115a2585690b/insights-operator/0.log" Jun 09 14:35:43.479510 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:43.479484 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-hf2bv_023aaede-6273-4afe-8d49-115a2585690b/insights-operator/1.log" Jun 09 14:35:43.499801 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:43.499780 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fl4nq_74529d19-d865-4dba-a40e-b7e63494eed2/kube-rbac-proxy/0.log" Jun 09 14:35:43.532516 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:43.532498 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fl4nq_74529d19-d865-4dba-a40e-b7e63494eed2/exporter/0.log" Jun 09 14:35:43.565878 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:43.565844 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fl4nq_74529d19-d865-4dba-a40e-b7e63494eed2/extractor/0.log" Jun 09 14:35:45.824223 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:45.824192 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8mncq/perf-node-gather-daemonset-d27hj" Jun 09 14:35:45.914924 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:45.914901 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-986494677-9pr9d_d3e182c6-611b-415e-bdc6-1cc1bd1b9d45/nginx/0.log" Jun 09 14:35:45.955146 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:45.955123 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-bnb54_56a1a2b7-48c2-4333-9f78-c5fb5e5d9ce0/manager/0.log" Jun 09 14:35:46.013455 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.013427 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-zppv6_44185177-68e6-4347-91e0-f4bf207f810a/manager/0.log" Jun 09 14:35:46.035969 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.035934 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c8bd7b74c-nwhpz_88138061-245f-49f5-ae80-3007e57a8eb3/maas-api/0.log" Jun 09 14:35:46.104484 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.104412 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-585c5cbb6c-txb8t_c634b229-f1eb-4a6b-97c7-83035be69d4a/manager/0.log" Jun 09 14:35:46.136332 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.136311 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-zltjm_2fe01e53-7e7f-4e81-ad29-ca40fa3af222/server/0.log" Jun 09 14:35:46.194234 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.194212 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-l8fbt_31edc12e-1f61-48f2-b550-628cd0e6b021/manager/0.log" Jun 09 14:35:46.216574 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.216545 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c57c599f5-26gnw_3fb57b60-574a-4454-8cf3-d6505fc762a3/manager/0.log" Jun 09 14:35:46.252499 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:46.252479 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c57c599f5-26gnw_3fb57b60-574a-4454-8cf3-d6505fc762a3/copy-manifests/0.log" Jun 09 14:35:52.598868 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:52.598836 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-2tn87_e08c5d6f-2a71-4d36-96cb-76aa5d5205b4/discovery/0.log" Jun 09 14:35:52.623189 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:52.623166 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df8999ffb-wff2z_fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c/kube-auth-proxy/0.log" Jun 09 14:35:52.678194 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:52.678165 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p9wg6_a5730f4a-2886-409d-b625-2e92ee477f88/payload-pre-processing/0.log" Jun 09 14:35:52.687696 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:52.687676 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/4.log" Jun 09 14:35:52.689892 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:52.689856 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/3.log" Jun 09 14:35:53.180739 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.180710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-mvc68_e9604d2b-81fc-4a1e-b664-ecd369cf2854/migrator/0.log" Jun 09 14:35:53.210837 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.210811 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-mvc68_e9604d2b-81fc-4a1e-b664-ecd369cf2854/graceful-termination/0.log" Jun 09 14:35:53.621984 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.621940 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-4m8dl_55cc31f5-c7f1-440b-92e1-ef3068e777e7/kube-storage-version-migrator-operator/1.log" Jun 09 14:35:53.622718 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.622701 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-4m8dl_55cc31f5-c7f1-440b-92e1-ef3068e777e7/kube-storage-version-migrator-operator/0.log" Jun 09 14:35:53.810656 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.810623 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-2tn87_e08c5d6f-2a71-4d36-96cb-76aa5d5205b4/discovery/0.log" Jun 09 14:35:53.841357 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.841331 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df8999ffb-wff2z_fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c/kube-auth-proxy/0.log" Jun 09 14:35:53.900037 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.899965 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p9wg6_a5730f4a-2886-409d-b625-2e92ee477f88/payload-pre-processing/0.log" Jun 09 14:35:53.915241 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.915218 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/4.log" Jun 09 14:35:53.917061 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:53.917038 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/3.log" Jun 09 14:35:55.190824 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.190756 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/kube-multus-additional-cni-plugins/0.log" Jun 09 14:35:55.227855 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.227830 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/egress-router-binary-copy/0.log" Jun 09 14:35:55.261980 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.261940 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/cni-plugins/0.log" Jun 09 14:35:55.293476 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.293447 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/bond-cni-plugin/0.log" Jun 09 14:35:55.327448 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.327423 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/routeoverride-cni/0.log" Jun 09 14:35:55.363182 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.363158 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/whereabouts-cni-bincopy/0.log" Jun 09 14:35:55.399991 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.399965 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mkqk7_2f6f3949-59cd-4c78-8709-b027e61dea18/whereabouts-cni/0.log" Jun 09 14:35:55.444526 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.444467 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-k7ngj_02228a0a-bfe5-4615-b4f2-45dfd3cd66e6/kube-multus/0.log" Jun 09 14:35:55.548493 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.548465 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-2tn87_e08c5d6f-2a71-4d36-96cb-76aa5d5205b4/discovery/0.log" Jun 09 14:35:55.586149 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.586121 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hsxpw_13916a76-8368-4bc7-b6d0-55615e3aab9e/network-metrics-daemon/0.log" Jun 09 14:35:55.628035 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.628011 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df8999ffb-wff2z_fb4c8b66-34c8-4ba6-8f94-4e8fdbe0870c/kube-auth-proxy/0.log" Jun 09 14:35:55.639106 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.639087 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hsxpw_13916a76-8368-4bc7-b6d0-55615e3aab9e/kube-rbac-proxy/0.log" Jun 09 14:35:55.685342 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.685317 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-p9wg6_a5730f4a-2886-409d-b625-2e92ee477f88/payload-pre-processing/0.log" Jun 09 14:35:55.694424 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.694348 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/4.log" Jun 09 14:35:55.696455 ip-10-0-128-214 kubenswrapper[2568]: I0609 14:35:55.696376 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-2rj9c_ff0c800c-89d1-42ab-9b21-91da52b54347/payload-processing/3.log"