Oct 30 18:30:41.812442 ip-10-0-130-102 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:30:42.245300 ip-10-0-130-102 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:42.245300 ip-10-0-130-102 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:30:42.245300 ip-10-0-130-102 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:42.245300 ip-10-0-130-102 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250350 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250436 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250439 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250443 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250446 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250449 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.250440 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250453 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250456 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250459 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250462 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250465 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250468 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250470 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250473 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250476 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250478 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250481 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250484 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250486 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250489 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250491 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250494 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250496 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250500 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250502 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250505 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.250692 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250507 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250512 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250516 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250521 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250524 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250530 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250533 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250536 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250538 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250541 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250544 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250546 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250548 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250551 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250555 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250557 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250560 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250563 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250566 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250568 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.251172 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250571 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250574 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250576 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250579 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250581 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250584 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250586 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250589 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250592 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250594 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250596 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250600 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250602 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250605 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250608 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250611 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250613 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250616 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250618 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250621 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.251648 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250624 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250627 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250630 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250633 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250635 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250639 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250645 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250649 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250652 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250655 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250658 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250661 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250664 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250667 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250670 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250673 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250675 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250680 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250683 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.252114 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250685 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250688 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250691 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250694 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250697 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250699 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250703 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250705 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250709 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250711 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250714 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250716 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250718 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250721 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250724 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250726 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250729 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250731 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250735 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.252580 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250738 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.250741 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251553 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251561 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251563 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251566 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251569 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251571 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251574 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251577 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251580 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251582 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251585 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251588 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251591 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251595 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251599 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251603 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251606 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251609 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.253050 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251611 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251615 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251617 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251620 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251623 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251625 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251628 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251631 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251633 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251636 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251639 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251642 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251645 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251647 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251650 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251653 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251656 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251659 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251662 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251664 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.253551 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251667 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251670 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251672 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251675 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251677 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251681 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251684 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251686 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251689 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251691 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251693 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251696 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251698 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251701 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251704 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251706 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251709 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251711 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.254016 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251714 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251718 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251722 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251725 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251728 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251731 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251734 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251736 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251739 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251742 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251744 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251747 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251750 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251752 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251755 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251758 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251760 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251763 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251765 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.254471 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251768 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251771 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251774 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251776 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251779 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251781 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251784 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251786 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251789 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251792 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251794 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251797 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251800 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251802 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251805 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251808 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251811 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251813 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251816 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251822 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.254913 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251824 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251827 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251830 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251832 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251835 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251838 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251841 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251843 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251846 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251848 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.251851 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252426 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252433 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252437 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252440 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252442 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252446 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252449 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252451 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.255403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252454 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252456 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252459 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252462 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252465 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252467 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252470 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252473 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252476 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252479 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252481 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252484 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252488 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252490 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252493 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252495 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252498 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252500 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252503 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252505 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.255863 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252508 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252510 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252512 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252515 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252518 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252520 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252523 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252526 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252528 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252531 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252533 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252536 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252538 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252541 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252543 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252546 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252548 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252551 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252553 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.256336 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252556 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252559 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252562 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252564 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252567 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252570 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252573 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252575 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252579 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252584 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252587 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252589 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252593 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252596 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252598 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252601 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252603 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252606 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252608 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.256795 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252611 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252613 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252616 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252619 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252621 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252624 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252626 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252629 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252632 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252634 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252637 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252639 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252644 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252646 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252649 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252651 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252654 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252656 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252659 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252662 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.257260 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252665 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252667 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252670 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252673 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252675 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252678 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252681 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252683 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252686 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252688 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252691 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252693 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252696 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252698 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252701 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252704 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252706 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252711 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252714 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.257745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.252717 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252825 2584 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252835 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252843 2584 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252848 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252854 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252858 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252864 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252868 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252871 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252874 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252878 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252882 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252885 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252888 2584 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252891 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252894 2584 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252897 2584 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252900 2584 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252904 2584 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252907 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252910 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252914 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252917 2584 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:30:42.258200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252921 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252924 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252927 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252931 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252934 2584 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252937 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252940 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252944 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252947 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252951 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252954 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252957 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252960 2584 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252967 2584 flags.go:64] FLAG: --enable-server="true" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252970 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252975 2584 flags.go:64] FLAG: --event-burst="100" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252978 2584 flags.go:64] FLAG: --event-qps="50" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252981 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252984 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252989 2584 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252994 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.252997 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253000 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253003 2584 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253006 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:30:42.258754 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253009 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253013 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253016 2584 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253018 2584 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253021 2584 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253024 2584 flags.go:64] FLAG: --feature-gates="" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253029 2584 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253032 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253035 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253038 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253041 2584 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253044 2584 flags.go:64] FLAG: --help="false" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253048 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253051 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253055 2584 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253058 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253061 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253065 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253068 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253071 2584 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253075 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253078 2584 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253082 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253085 2584 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:30:42.259372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253088 2584 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253091 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253095 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253099 2584 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253101 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253105 2584 flags.go:64] FLAG: --lock-file="" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253108 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253111 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253114 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253121 2584 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253124 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253127 2584 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253130 2584 flags.go:64] FLAG: --logging-format="text" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253133 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253137 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253140 2584 flags.go:64] FLAG: --manifest-url="" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253158 2584 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253163 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253167 2584 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253171 2584 flags.go:64] FLAG: --max-pods="110" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253174 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253177 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253181 2584 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253184 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253187 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:30:42.259966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253190 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253193 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253201 2584 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253205 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253208 2584 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253211 2584 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253214 2584 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253217 2584 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253220 2584 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253226 2584 flags.go:64] FLAG: --port="10250" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253229 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253232 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076af53a62a845815" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253235 2584 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253238 2584 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253241 2584 flags.go:64] FLAG: --register-node="true" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253245 2584 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253252 2584 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253255 2584 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253258 2584 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253261 2584 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253265 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253268 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253271 2584 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253274 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253277 2584 flags.go:64] FLAG: --runonce="false" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253280 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:30:42.260563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253283 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253286 2584 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253289 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253292 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253296 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253299 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253302 2584 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253306 2584 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253309 2584 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253313 2584 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253316 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253320 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253323 2584 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253326 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253331 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253335 2584 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253338 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253342 2584 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253345 2584 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253348 2584 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253352 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253356 2584 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253361 2584 flags.go:64] FLAG: --v="2" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253365 2584 flags.go:64] FLAG: --version="false" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253370 2584 flags.go:64] FLAG: --vmodule="" Oct 30 18:30:42.261242 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253374 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253378 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253521 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253525 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253528 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253531 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253534 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253537 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253539 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253542 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253545 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253548 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253551 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253554 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253557 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253559 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253563 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253566 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253568 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.261911 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253571 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253574 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253577 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253581 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253584 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253587 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253589 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253592 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253594 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253597 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253600 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253603 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253606 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253608 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253611 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253614 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253616 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253619 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253621 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.262377 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253624 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253626 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253629 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253632 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253634 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253637 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253639 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253643 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253646 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253648 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253651 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253655 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253659 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253662 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253666 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253669 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253673 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253676 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253679 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253682 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.262830 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253685 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253687 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253690 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253692 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253695 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253698 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253700 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253703 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253706 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253708 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253711 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253714 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253716 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253719 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253722 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253724 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253727 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253729 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253732 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.263321 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253735 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253738 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253740 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253743 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253746 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253748 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253751 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253753 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253756 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253759 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253762 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253765 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253767 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253770 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253773 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253775 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253778 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253780 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253783 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253786 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.263771 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253788 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253791 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253793 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253797 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253802 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253805 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253808 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253811 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253813 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253816 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253818 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.253844 2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253881 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253885 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253888 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253892 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.264263 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253895 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253897 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253900 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253903 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253906 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253908 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253912 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253915 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253917 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253920 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253923 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253925 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253928 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253931 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253933 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253937 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253939 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253943 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253947 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253950 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.264639 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253954 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253957 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253960 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253962 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253965 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253968 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253970 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253973 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253975 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253978 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253982 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253985 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253988 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253991 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253993 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253996 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.253998 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254001 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254005 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.265098 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254007 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254010 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254013 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254016 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254018 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254021 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254024 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254027 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254030 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254033 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254035 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254038 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254041 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254043 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254046 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254049 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254051 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254054 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254057 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.265709 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254059 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254062 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254065 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254068 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254071 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254074 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254080 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254083 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254085 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254088 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254091 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254093 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254097 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254100 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254104 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254107 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254110 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254113 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254116 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.266183 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254119 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254122 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254124 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254127 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254129 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254132 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254135 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254137 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254140 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254156 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254159 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254162 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254165 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254167 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254170 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254173 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254175 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254179 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254182 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254185 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.266681 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254188 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254191 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254194 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254197 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.254199 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.261433 2584 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.261448 2584 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.262181 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.262188 2584 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262247 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262253 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262257 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262260 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262263 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262266 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262268 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262271 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262274 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262276 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262279 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.267175 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262282 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262285 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262289 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262293 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262296 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262299 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262302 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262304 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262307 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262310 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262313 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262315 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262319 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262322 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262324 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262327 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262330 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262333 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262336 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.267662 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262339 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262341 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262344 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262347 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262349 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262352 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262355 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262357 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262360 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262363 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262365 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262368 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262371 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262374 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262376 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262379 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262382 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262385 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262388 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262391 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.268096 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262393 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262396 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262399 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262402 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262405 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262407 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262411 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262413 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262416 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262418 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262421 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262424 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262427 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262429 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262432 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262435 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262437 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262440 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262442 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262445 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.268570 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262447 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262450 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262452 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262455 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262458 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262461 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262463 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262466 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262468 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262471 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262473 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262476 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262479 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262481 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262484 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262487 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262490 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262493 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262495 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.269021 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262499 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262501 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262504 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262509 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262512 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262515 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262518 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262521 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262524 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262526 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262529 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262532 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262534 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262537 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262539 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262542 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.269540 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262544 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.262572 2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262715 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262719 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262722 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262726 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262728 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262731 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262734 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262736 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262739 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262742 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262745 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262748 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262750 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.269948 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262753 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262756 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262759 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262761 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262764 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262767 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262769 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262772 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262774 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262778 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262782 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262785 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262788 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262791 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262793 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262796 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262799 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262801 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262804 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262807 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.270497 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262809 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262812 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262815 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262817 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262820 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262822 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262824 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262827 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262830 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262833 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262837 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262841 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262844 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262848 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262850 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262854 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262857 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262860 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262863 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.271286 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262865 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262868 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262870 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262873 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262875 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262878 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262880 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262883 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262886 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262888 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262891 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262893 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262896 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262898 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262901 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262903 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262906 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262908 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262911 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.271849 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262913 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262916 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262918 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262921 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262924 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262926 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262929 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262932 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262934 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262937 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262940 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262943 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262946 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262948 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262951 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262954 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262956 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262959 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262961 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262964 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.272391 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262966 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262969 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262972 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262974 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262976 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262979 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262982 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262984 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262987 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262989 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262992 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262995 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.262997 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263000 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263002 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.263025 2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:30:42.272854 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263055 2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263059 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263062 2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263065 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263068 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263072 2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263076 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263080 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263084 2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263086 2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263089 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263092 2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263094 2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263097 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263100 2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263102 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263105 2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263108 2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263110 2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263113 2584 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:30:42.273264 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263115 2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263118 2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263120 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263123 2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263125 2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263128 2584 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263132 2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263135 2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263139 2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263153 2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263157 2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263159 2584 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263162 2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263165 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263168 2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263171 2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263174 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263176 2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263179 2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:30:42.273750 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263181 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263185 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263187 2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263190 2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263192 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263195 2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263197 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263200 2584 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263203 2584 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263205 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263208 2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263210 2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263213 2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263215 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263218 2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263220 2584 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263223 2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263225 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263228 2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:30:42.274202 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263231 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263233 2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263235 2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263238 2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263240 2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263243 2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263248 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263252 2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263254 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263257 2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263259 2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263262 2584 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263265 2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263267 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263270 2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263276 2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263279 2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263282 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263285 2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263288 2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:30:42.274632 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263290 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263293 2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263295 2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263298 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263300 2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263303 2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263305 2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263308 2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263311 2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263313 2584 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263316 2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263318 2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263322 2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263324 2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263327 2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263330 2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263332 2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263335 2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263339 2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:30:42.275136 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263342 2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263345 2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263348 2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263352 2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263355 2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263357 2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263360 2584 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263363 2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:42.263365 2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.263416 2584 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.265412 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.266370 2584 server.go:1014] "Starting client certificate rotation" Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.266467 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.266496 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.269440 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:30:42.275601 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.273430 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:30:42.285292 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.285277 2584 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:30:42.287652 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.287638 2584 log.go:25] "Validated CRI v1 image API" Oct 30 18:30:42.288522 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.288498 2584 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:30:42.292579 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.292554 2584 fs.go:135] Filesystem UUIDs: map[022651ab-6fe0-445f-9e4b-cf87ff853eb2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a8176a86-0e55-4cd7-b5a8-b03c227dde75:/dev/nvme0n1p4] Oct 30 18:30:42.292631 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.292579 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 18:30:42.297739 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.297624 2584 manager.go:217] Machine: {Timestamp:2026-10-30 18:30:42.296270393 +0000 UTC m=+0.355389389 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098914 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cc837afda1383576da86a9fc14cc8 SystemUUID:ec2cc837-afda-1383-576d-a86a9fc14cc8 BootID:cc18319c-eabf-43a4-89c2-8671d813ec96 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f5:40:cf:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f5:40:cf:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:a8:eb:36:20:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:30:42.297817 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.297741 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:30:42.297845 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.297818 2584 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:30:42.300548 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.300522 2584 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:30:42.300677 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.300550 2584 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:30:42.300761 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.300679 2584 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:30:42.300761 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.300688 2584 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:30:42.300761 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.300731 2584 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:30:42.300761 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.300750 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:30:42.301593 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.301582 2584 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:30:42.306246 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.306234 2584 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:30:42.306366 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.306356 2584 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:30:42.307186 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.307177 2584 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:30:42.307225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.307190 2584 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:30:42.307225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.307203 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:30:42.307225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.307212 2584 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:30:42.307225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.307220 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:30:42.307566 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.307553 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:30:42.310731 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.310694 2584 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:30:42.316930 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.316912 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:30:42.317940 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.317928 2584 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:30:42.317979 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.317959 2584 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:30:42.318117 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318108 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:30:42.318161 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318123 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:30:42.318161 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318130 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:30:42.318161 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318136 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:30:42.318161 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318160 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:30:42.318275 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318167 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:30:42.318715 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318706 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:30:42.318743 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318715 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:30:42.318743 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318723 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:30:42.318743 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318729 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:30:42.318743 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318737 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:30:42.318901 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.318747 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:30:42.319477 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.319467 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:30:42.319507 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.319481 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:30:42.323295 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.323283 2584 server.go:1263] "Started kubelet" Oct 30 18:30:42.323431 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.323374 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:30:42.323502 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.323488 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:30:42.323568 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.323545 2584 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:30:42.324135 ip-10-0-130-102 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:30:42.325845 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.325754 2584 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:30:42.325845 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.325805 2584 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:30:42.329416 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.329400 2584 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 18:30:42.329744 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.329731 2584 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:30:42.329795 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.329745 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:30:42.329856 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.329833 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.330350 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330286 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:30:42.330350 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330300 2584 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:30:42.330350 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330313 2584 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:30:42.330503 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330401 2584 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:30:42.330503 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330410 2584 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:30:42.330768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330748 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:30:42.330853 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330769 2584 factory.go:55] Registering systemd factory Oct 30 18:30:42.330853 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330779 2584 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:30:42.330997 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330978 2584 factory.go:153] Registering CRI-O factory Oct 30 18:30:42.330997 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.330995 2584 factory.go:223] Registration of the crio container factory successfully Oct 30 18:30:42.331182 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.331015 2584 factory.go:103] Registering Raw factory Oct 30 18:30:42.331182 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.331033 2584 manager.go:1199] Started watching for new ooms in manager Oct 30 18:30:42.331897 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.331886 2584 manager.go:319] Starting recovery of all containers Oct 30 18:30:42.338125 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.338085 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:30:42.342497 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.342473 2584 manager.go:324] Recovery completed Oct 30 18:30:42.349784 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.349767 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.351844 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.351821 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.351921 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.351862 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.351921 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.351877 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.352535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.352511 2584 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:30:42.352535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.352523 2584 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:30:42.352654 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.352543 2584 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:30:42.352654 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.352557 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.353350 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.353332 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.353408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.353369 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.353408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.353386 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.354840 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.354827 2584 policy_none.go:50] "Start" Oct 30 18:30:42.354898 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.354844 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:30:42.354898 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.354858 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:30:42.356928 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.356914 2584 policy_none.go:44] "Start" Oct 30 18:30:42.357340 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.357311 2584 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 18:30:42.357449 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.357313 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 18:30:42.358450 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.357318 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-102.ec2.internal.18e36231b92d3b52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-102.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-102.ec2.internal,},FirstTimestamp:2026-10-30 18:30:42.323258194 +0000 UTC m=+0.382377190,LastTimestamp:2026-10-30 18:30:42.323258194 +0000 UTC m=+0.382377190,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-102.ec2.internal,}" Oct 30 18:30:42.364742 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.364718 2584 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:30:42.374934 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.374853 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-102.ec2.internal.18e36231bae16d88 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-102.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-102.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-102.ec2.internal,},FirstTimestamp:2026-10-30 18:30:42.351844744 +0000 UTC m=+0.410963750,LastTimestamp:2026-10-30 18:30:42.351844744 +0000 UTC m=+0.410963750,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-102.ec2.internal,}" Oct 30 18:30:42.375038 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.374929 2584 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 18:30:42.375038 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.375011 2584 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.387502 2584 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.394732 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-102.ec2.internal.18e36231bae1ca27 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-102.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-102.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-102.ec2.internal,},FirstTimestamp:2026-10-30 18:30:42.351868455 +0000 UTC m=+0.410987452,LastTimestamp:2026-10-30 18:30:42.351868455 +0000 UTC m=+0.410987452,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-102.ec2.internal,}" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.395760 2584 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.396597 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.396620 2584 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.396640 2584 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.396678 2584 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.400701 2584 manager.go:341] "Starting Device Plugin manager" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.400727 2584 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.400736 2584 server.go:87] "Starting device plugin registration server" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.400944 2584 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.400956 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.401033 2584 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.401112 2584 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.401122 2584 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.401544 2584 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:30:42.407008 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.401596 2584 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.411731 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.411122 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-102.ec2.internal.18e36231bae1ff52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-102.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-102.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-102.ec2.internal,},FirstTimestamp:2026-10-30 18:30:42.351882066 +0000 UTC m=+0.411001063,LastTimestamp:2026-10-30 18:30:42.351882066 +0000 UTC m=+0.411001063,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-102.ec2.internal,}" Oct 30 18:30:42.414003 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.413984 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2hf7g" Oct 30 18:30:42.426534 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.426512 2584 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" Oct 30 18:30:42.426598 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.426505 2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-102.ec2.internal.18e36231bae16d88\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-102.ec2.internal.18e36231bae16d88 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-102.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-102.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-102.ec2.internal,},FirstTimestamp:2026-10-30 18:30:42.351844744 +0000 UTC m=+0.410963750,LastTimestamp:2026-10-30 18:30:42.353351848 +0000 UTC m=+0.412470848,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-102.ec2.internal,}" Oct 30 18:30:42.431488 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.431471 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2hf7g" Oct 30 18:30:42.446426 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.446346 2584 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-102.ec2.internal.18e36231bae1ca27\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-102.ec2.internal.18e36231bae1ca27 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-102.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-102.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-102.ec2.internal,},FirstTimestamp:2026-10-30 18:30:42.351868455 +0000 UTC m=+0.410987452,LastTimestamp:2026-10-30 18:30:42.353376201 +0000 UTC m=+0.412495200,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-102.ec2.internal,}" Oct 30 18:30:42.497133 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.497045 2584 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal"] Oct 30 18:30:42.497133 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.497108 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.498096 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.498077 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.498190 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.498108 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.498190 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.498119 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.500378 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.500366 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.501034 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501016 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.501158 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501055 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.501158 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501079 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.501158 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501093 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.501158 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501056 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.501158 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501140 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.501731 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501717 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.501820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501729 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.501820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501747 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.501820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501752 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.501820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501762 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.501820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501762 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.501820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.501801 2584 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.503166 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.503136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.503212 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.503186 2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:30:42.503920 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.503905 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:30:42.504025 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.503926 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:30:42.504025 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.503939 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:30:42.523511 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.523495 2584 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.523601 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.523515 2584 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-102.ec2.internal\": node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.536418 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.536401 2584 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-102.ec2.internal\" not found" node="ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.539409 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.539392 2584 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-102.ec2.internal\" not found" node="ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.560393 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.560372 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.630608 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.630574 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.630608 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.630602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdb7686e846fff2d163fb7282f45a982-config\") pod \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" (UID: \"bdb7686e846fff2d163fb7282f45a982\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.630794 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.630621 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.661275 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.661249 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.731304 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.731259 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.731304 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.731282 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.731501 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.731329 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.731501 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.731346 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdb7686e846fff2d163fb7282f45a982-config\") pod \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" (UID: \"bdb7686e846fff2d163fb7282f45a982\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.731501 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.731372 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdb7686e846fff2d163fb7282f45a982-config\") pod \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" (UID: \"bdb7686e846fff2d163fb7282f45a982\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.731501 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.731391 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.762353 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.762283 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.839620 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.839584 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.842760 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:42.842732 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:42.862915 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.862709 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:42.963631 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:42.963594 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:43.064174 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.064084 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:43.165494 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.165457 2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:30:43.207151 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.207125 2584 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:43.230951 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.230927 2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:30:43.260455 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.260433 2584 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:30:43.262695 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.262678 2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:43.266610 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.266590 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 18:30:43.266725 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.266708 2584 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 18:30:43.266813 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.266784 2584 kubelet.go:3370] "Failed creating a mirror pod" err="Post \"https://ad1c9def951c3486c987d25017e9e9cc-3acd125417877706.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.130.102:34948->3.12.206.126:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:43.307950 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.307921 2584 apiserver.go:52] "Watching apiserver" Oct 30 18:30:43.330221 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.330131 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 18:30:43.344147 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.344116 2584 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:43.381839 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.381810 2584 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:30:43.382152 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.382129 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-plcdm","openshift-image-registry/node-ca-9nr4w","openshift-multus/network-metrics-daemon-xlm58","kube-system/global-pull-secret-syncer-jw7nd","openshift-multus/multus-additional-cni-plugins-jjvbp","openshift-network-operator/iptables-alerter-l5zgj","openshift-ovn-kubernetes/ovnkube-node-l6tz8","openshift-cluster-node-tuning-operator/tuned-7lgct","openshift-network-diagnostics/network-check-target-gwzzt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz","openshift-dns/node-resolver-sm8gj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal","openshift-multus/multus-fxrmc"] Oct 30 18:30:43.384325 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.384306 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.386489 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.386471 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.388468 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.388453 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:43.388535 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.388507 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:43.391286 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.391271 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.393591 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.393572 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.396383 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.396364 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.399440 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.399422 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.401675 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.401652 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.401775 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.401692 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.401775 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.401761 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xfxj4\"" Oct 30 18:30:43.402372 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.402357 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.402449 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.402424 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:43.403202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.403187 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.403353 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.403336 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:30:43.403353 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.403346 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ft7c\"" Oct 30 18:30:43.403445 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.403342 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.405455 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.405437 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:43.405544 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.405525 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:43.408712 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.408697 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.411214 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.411192 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.413951 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.413929 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.415184 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.415164 2584 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:30:43.415296 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.415229 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ckg4c" Oct 30 18:30:43.431401 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.431385 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:30:43.434499 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434482 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/399ea207-fdb0-4a0a-a841-b113cffc4c3e-iptables-alerter-script\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.434586 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434513 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-dbus\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.434645 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434606 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-socket-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.434695 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434640 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-socket-dir-parent\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.434695 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434681 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-kubelet\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.434788 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434706 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-multus-certs\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.434788 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434767 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-conf\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.434865 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434790 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-run\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.434865 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434806 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-host\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.434865 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434853 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7rmv\" (UniqueName: \"kubernetes.io/projected/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-kube-api-access-f7rmv\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.435056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434881 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-hostroot\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.435056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434900 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-systemd-units\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-systemd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.434998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-var-lib-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435029 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-netd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435049 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-config\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435073 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysconfig\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463c6228-727e-4d4f-9b80-1db615657b4f-tmp-dir\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435127 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-node-log\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435196 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87hnt\" (UniqueName: \"kubernetes.io/projected/4bb762ff-7409-4218-8421-d0b376a46f98-kube-api-access-87hnt\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435234 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435268 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-systemd\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435285 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-daemon-config\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.435322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435310 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd977\" (UniqueName: \"kubernetes.io/projected/463c6228-727e-4d4f-9b80-1db615657b4f-kube-api-access-fd977\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435333 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-host\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435353 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435383 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-registration-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435454 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-cnibin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435504 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-lib-modules\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435532 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgmhk\" (UniqueName: \"kubernetes.io/projected/b60126a0-255e-4b6d-9166-18c92777ddc5-kube-api-access-hgmhk\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435557 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-etc-kubernetes\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435582 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:43.435632 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435620 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec318-a75a-4dd0-9da0-ffc41310b641-agent-certs\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435640 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-system-cni-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435692 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435712 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435726 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/463c6228-727e-4d4f-9b80-1db615657b4f-hosts-file\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435755 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvpks\" (UniqueName: \"kubernetes.io/projected/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-kube-api-access-fvpks\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435790 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435830 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-device-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435834 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:25:42 +0000 UTC" deadline="2028-08-14 15:54:54.12837643 +0000 UTC" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435862 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15693h24m10.692518509s" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435867 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.435981 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-netns\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.435985 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw8z6\" (UniqueName: \"kubernetes.io/projected/cf9ba972-af6a-4cda-a59b-f51874d49aae-kube-api-access-dw8z6\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436022 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-tuned\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-bin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436085 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-var-lib-kubelet\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436124 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-tmp\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436224 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-bin\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436258 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436295 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-multus\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.436396 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436322 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-modprobe-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436402 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-kubernetes\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436444 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b4ec318-a75a-4dd0-9da0-ffc41310b641-konnectivity-ca\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436500 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-kubelet-config\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436525 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-serviceca\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436556 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd8rj\" (UniqueName: \"kubernetes.io/projected/399ea207-fdb0-4a0a-a841-b113cffc4c3e-kube-api-access-fd8rj\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436609 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436637 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-script-lib\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436671 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-etc-selinux\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436693 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-sys-fs\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436709 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-k8s-cni-cncf-io\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436735 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-cni-binary-copy\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.436747 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436751 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-slash\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436794 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-etc-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436825 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-ovn\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436857 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-env-overrides\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436877 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovn-node-metrics-cert\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436893 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-sys\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436931 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cnibin\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.436978 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-netns\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437004 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zhgq\" (UniqueName: \"kubernetes.io/projected/47df06d1-8c08-47c2-9e1f-10636f6bf035-kube-api-access-8zhgq\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-system-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437057 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437096 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437137 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/399ea207-fdb0-4a0a-a841-b113cffc4c3e-host-slash\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-kubelet\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437223 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-log-socket\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437246 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.437281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437267 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-conf-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.437773 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.437314 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zshw\" (UniqueName: \"kubernetes.io/projected/10ed4e5d-002c-48ca-ad1e-199694eef36b-kube-api-access-5zshw\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:43.464475 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.464443 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d214f128413268df5e715b75027209.slice/crio-13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726 WatchSource:0}: Error finding container 13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726: Status 404 returned error can't find the container with id 13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726 Oct 30 18:30:43.464730 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.464709 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb7686e846fff2d163fb7282f45a982.slice/crio-cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189 WatchSource:0}: Error finding container cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189: Status 404 returned error can't find the container with id cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189 Oct 30 18:30:43.469134 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.469116 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:30:43.511208 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511166 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:30:43.511407 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511277 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:30:43.511407 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511288 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:30:43.511589 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511510 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.511589 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511530 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.511679 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511612 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.511679 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511608 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:30:43.511915 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511886 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:30:43.511915 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511898 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.511915 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511891 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ghgqv\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511932 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-56ppb\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.511934 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512010 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512070 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512105 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512107 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsmgf\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512132 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.512181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512135 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:30:43.512468 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.512387 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.513639 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.513608 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtzwt\"" Oct 30 18:30:43.513714 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.513700 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:30:43.513768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.513754 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8srjr\"" Oct 30 18:30:43.513815 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.513763 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:30:43.525753 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.525734 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:30:43.525827 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.525750 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kkmm\"" Oct 30 18:30:43.525827 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.525752 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k7jl9\"" Oct 30 18:30:43.525827 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.525771 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:30:43.525827 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.525810 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:30:43.526346 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.526330 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:30:43.538512 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538492 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/399ea207-fdb0-4a0a-a841-b113cffc4c3e-iptables-alerter-script\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.538582 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538516 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-dbus\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.538582 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538532 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-socket-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.538582 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538548 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-socket-dir-parent\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.538582 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538562 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-kubelet\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.538582 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538578 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-multus-certs\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-conf\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538666 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-multus-certs\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538693 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-kubelet\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538714 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-dbus\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538722 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-run\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538713 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-socket-dir-parent\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538745 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-socket-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538786 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-host\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538792 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-run\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538818 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7rmv\" (UniqueName: \"kubernetes.io/projected/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-kube-api-access-f7rmv\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.538850 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538844 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-hostroot\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538840 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-host\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538870 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-systemd-units\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538901 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-systemd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538906 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-systemd-units\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538907 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-hostroot\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538853 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-conf\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538934 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-var-lib-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538969 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-netd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538974 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-systemd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.538993 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-config\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539011 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-netd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539009 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-var-lib-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539020 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysconfig\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539058 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463c6228-727e-4d4f-9b80-1db615657b4f-tmp-dir\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539075 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysconfig\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539085 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-node-log\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539113 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87hnt\" (UniqueName: \"kubernetes.io/projected/4bb762ff-7409-4218-8421-d0b376a46f98-kube-api-access-87hnt\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.539408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539140 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539191 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-systemd\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539156 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-node-log\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539240 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-daemon-config\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539270 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd977\" (UniqueName: \"kubernetes.io/projected/463c6228-727e-4d4f-9b80-1db615657b4f-kube-api-access-fd977\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539285 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-systemd\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539296 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-host\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539308 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/399ea207-fdb0-4a0a-a841-b113cffc4c3e-iptables-alerter-script\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539327 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539358 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-registration-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539373 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-host\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539383 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-cnibin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539410 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463c6228-727e-4d4f-9b80-1db615657b4f-tmp-dir\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-lib-modules\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539327 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539492 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgmhk\" (UniqueName: \"kubernetes.io/projected/b60126a0-255e-4b6d-9166-18c92777ddc5-kube-api-access-hgmhk\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.539522 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:43.540173 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539564 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-cnibin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.539597 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret podName:deaa4bf3-e4f9-4b22-9bda-eba0080c0697 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:44.0395762 +0000 UTC m=+2.098695222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret") pod "global-pull-secret-syncer-jw7nd" (UID: "deaa4bf3-e4f9-4b22-9bda-eba0080c0697") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-etc-kubernetes\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539632 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-registration-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539640 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-lib-modules\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539669 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-config\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539687 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-etc-kubernetes\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539655 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539727 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec318-a75a-4dd0-9da0-ffc41310b641-agent-certs\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-system-cni-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.539736 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539791 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539827 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-system-cni-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539909 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539920 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-daemon-config\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.539946 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/463c6228-727e-4d4f-9b80-1db615657b4f-hosts-file\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.541068 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.539968 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:30:44.039940145 +0000 UTC m=+2.099059132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540004 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvpks\" (UniqueName: \"kubernetes.io/projected/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-kube-api-access-fvpks\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540014 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/463c6228-727e-4d4f-9b80-1db615657b4f-hosts-file\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540031 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540074 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-device-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540088 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540103 2584 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540115 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540170 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-netns\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw8z6\" (UniqueName: \"kubernetes.io/projected/cf9ba972-af6a-4cda-a59b-f51874d49aae-kube-api-access-dw8z6\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540202 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-device-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540205 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-tuned\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540249 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-netns\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540253 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-bin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540290 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-var-lib-kubelet\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540294 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-bin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540315 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-tmp\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.542202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540349 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-var-lib-kubelet\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540354 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540382 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-bin\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540409 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540434 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-multus\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540445 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540461 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-modprobe-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540492 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540498 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-kubernetes\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540497 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-multus\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540450 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-bin\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540549 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-modprobe-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540529 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b4ec318-a75a-4dd0-9da0-ffc41310b641-konnectivity-ca\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540576 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-kubernetes\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540593 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-kubelet-config\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540623 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-serviceca\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540638 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-kubelet-config\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:43.543201 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540650 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd8rj\" (UniqueName: \"kubernetes.io/projected/399ea207-fdb0-4a0a-a841-b113cffc4c3e-kube-api-access-fd8rj\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540678 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-script-lib\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540734 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-etc-selinux\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540759 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-sys-fs\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540785 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-k8s-cni-cncf-io\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540810 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-cni-binary-copy\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540838 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-slash\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540866 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-etc-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.540907 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541001 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-k8s-cni-cncf-io\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-ovn\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541045 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-env-overrides\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovn-node-metrics-cert\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541089 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b4ec318-a75a-4dd0-9da0-ffc41310b641-konnectivity-ca\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541140 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-serviceca\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541146 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-slash\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544076 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541096 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-sys\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541183 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-ovn\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cnibin\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541201 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541093 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-etc-selinux\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541259 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cnibin\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541264 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-netns\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541267 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-sys-fs\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541291 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zhgq\" (UniqueName: \"kubernetes.io/projected/47df06d1-8c08-47c2-9e1f-10636f6bf035-kube-api-access-8zhgq\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541311 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-system-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541327 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541341 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-sys\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541355 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541391 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-etc-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541392 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-system-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541435 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-netns\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541477 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/399ea207-fdb0-4a0a-a841-b113cffc4c3e-host-slash\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.544732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541526 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-kubelet\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541563 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-log-socket\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541611 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541642 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-conf-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541668 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zshw\" (UniqueName: \"kubernetes.io/projected/10ed4e5d-002c-48ca-ad1e-199694eef36b-kube-api-access-5zshw\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541884 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-kubelet\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541919 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-log-socket\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541953 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541602 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-env-overrides\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.541992 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-conf-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.542937 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.543184 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-script-lib\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.543187 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/399ea207-fdb0-4a0a-a841-b113cffc4c3e-host-slash\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.544246 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-cni-binary-copy\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.544548 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-tmp\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.544562 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-tuned\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.544670 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovn-node-metrics-cert\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.545277 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.544764 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec318-a75a-4dd0-9da0-ffc41310b641-agent-certs\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.545787 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.545493 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.571485 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.571417 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:43.571485 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.571448 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:43.571485 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.571463 2584 projected.go:196] Error preparing data for projected volume kube-api-access-j6qqs for pod openshift-network-diagnostics/network-check-target-gwzzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:43.571700 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:43.571540 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs podName:4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:44.071518769 +0000 UTC m=+2.130637774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j6qqs" (UniqueName: "kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs") pod "network-check-target-gwzzt" (UID: "4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:43.571845 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.571808 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87hnt\" (UniqueName: \"kubernetes.io/projected/4bb762ff-7409-4218-8421-d0b376a46f98-kube-api-access-87hnt\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.573796 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.573766 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvpks\" (UniqueName: \"kubernetes.io/projected/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-kube-api-access-fvpks\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.574771 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.574746 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd8rj\" (UniqueName: \"kubernetes.io/projected/399ea207-fdb0-4a0a-a841-b113cffc4c3e-kube-api-access-fd8rj\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.574988 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.574963 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgmhk\" (UniqueName: \"kubernetes.io/projected/b60126a0-255e-4b6d-9166-18c92777ddc5-kube-api-access-hgmhk\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.575902 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.575878 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7rmv\" (UniqueName: \"kubernetes.io/projected/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-kube-api-access-f7rmv\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.576411 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.576282 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zshw\" (UniqueName: \"kubernetes.io/projected/10ed4e5d-002c-48ca-ad1e-199694eef36b-kube-api-access-5zshw\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:43.576526 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.576503 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw8z6\" (UniqueName: \"kubernetes.io/projected/cf9ba972-af6a-4cda-a59b-f51874d49aae-kube-api-access-dw8z6\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.577143 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.577123 2584 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:43.577573 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.577550 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd977\" (UniqueName: \"kubernetes.io/projected/463c6228-727e-4d4f-9b80-1db615657b4f-kube-api-access-fd977\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.578958 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.578926 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zhgq\" (UniqueName: \"kubernetes.io/projected/47df06d1-8c08-47c2-9e1f-10636f6bf035-kube-api-access-8zhgq\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.705662 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.705623 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:30:43.716078 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.716048 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60126a0_255e_4b6d_9166_18c92777ddc5.slice/crio-34567c3b2e7101c112115bbafd7b5710e7d212377e56311d8bb00897be55801f WatchSource:0}: Error finding container 34567c3b2e7101c112115bbafd7b5710e7d212377e56311d8bb00897be55801f: Status 404 returned error can't find the container with id 34567c3b2e7101c112115bbafd7b5710e7d212377e56311d8bb00897be55801f Oct 30 18:30:43.732506 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.732482 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:30:43.740829 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.740805 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:43.741212 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.741190 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d10c3e_223f_41ef_ac3f_cd428bf43ebd.slice/crio-741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e WatchSource:0}: Error finding container 741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e: Status 404 returned error can't find the container with id 741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e Oct 30 18:30:43.749456 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.749436 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4ec318_a75a_4dd0_9da0_ffc41310b641.slice/crio-ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843 WatchSource:0}: Error finding container ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843: Status 404 returned error can't find the container with id ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843 Oct 30 18:30:43.775839 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.775816 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:30:43.778114 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.778096 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:30:43.787668 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.787630 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod739cc6ce_77da_492c_9cfc_6bfe5c50dd56.slice/crio-95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d WatchSource:0}: Error finding container 95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d: Status 404 returned error can't find the container with id 95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d Oct 30 18:30:43.788262 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.788245 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399ea207_fdb0_4a0a_a841_b113cffc4c3e.slice/crio-2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf WatchSource:0}: Error finding container 2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf: Status 404 returned error can't find the container with id 2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf Oct 30 18:30:43.799031 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.799009 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:30:43.799192 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.799171 2584 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:43.801395 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.801376 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:30:43.804131 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.804116 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fxrmc" Oct 30 18:30:43.810249 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:43.808856 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:30:43.813818 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.813722 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463c6228_727e_4d4f_9b80_1db615657b4f.slice/crio-edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874 WatchSource:0}: Error finding container edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874: Status 404 returned error can't find the container with id edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874 Oct 30 18:30:43.817214 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.817195 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb762ff_7409_4218_8421_d0b376a46f98.slice/crio-77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04 WatchSource:0}: Error finding container 77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04: Status 404 returned error can't find the container with id 77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04 Oct 30 18:30:43.818532 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.818511 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9ba972_af6a_4cda_a59b_f51874d49aae.slice/crio-dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66 WatchSource:0}: Error finding container dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66: Status 404 returned error can't find the container with id dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66 Oct 30 18:30:43.821823 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:30:43.821799 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47df06d1_8c08_47c2_9e1f_10636f6bf035.slice/crio-f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880 WatchSource:0}: Error finding container f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880: Status 404 returned error can't find the container with id f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880 Oct 30 18:30:44.044996 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.044952 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:44.045205 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.045007 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:44.045205 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.045181 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:44.045328 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.045259 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:30:45.045239856 +0000 UTC m=+3.104358856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:44.045596 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.045180 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:44.045698 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.045628 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret podName:deaa4bf3-e4f9-4b22-9bda-eba0080c0697 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:45.045610854 +0000 UTC m=+3.104729856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret") pod "global-pull-secret-syncer-jw7nd" (UID: "deaa4bf3-e4f9-4b22-9bda-eba0080c0697") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:44.146119 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.146015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:44.146314 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.146250 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:44.146314 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.146275 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:44.146314 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.146289 2584 projected.go:196] Error preparing data for projected volume kube-api-access-j6qqs for pod openshift-network-diagnostics/network-check-target-gwzzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:44.146497 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:44.146350 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs podName:4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:45.146331663 +0000 UTC m=+3.205450650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6qqs" (UniqueName: "kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs") pod "network-check-target-gwzzt" (UID: "4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:44.419295 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.419180 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:25:43 +0000 UTC" deadline="2028-05-03 06:21:09.628221283 +0000 UTC" Oct 30 18:30:44.419295 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.419243 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13211h50m25.208982255s" Oct 30 18:30:44.425751 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.425441 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9nr4w" event={"ID":"11d10c3e-223f-41ef-ac3f-cd428bf43ebd","Type":"ContainerStarted","Data":"741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e"} Oct 30 18:30:44.431144 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.431021 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880"} Oct 30 18:30:44.439345 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.439304 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sm8gj" event={"ID":"463c6228-727e-4d4f-9b80-1db615657b4f","Type":"ContainerStarted","Data":"edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874"} Oct 30 18:30:44.442820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.442698 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l5zgj" event={"ID":"399ea207-fdb0-4a0a-a841-b113cffc4c3e","Type":"ContainerStarted","Data":"2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf"} Oct 30 18:30:44.462422 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.462380 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-plcdm" event={"ID":"8b4ec318-a75a-4dd0-9da0-ffc41310b641","Type":"ContainerStarted","Data":"ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843"} Oct 30 18:30:44.473998 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.473963 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7lgct" event={"ID":"b60126a0-255e-4b6d-9166-18c92777ddc5","Type":"ContainerStarted","Data":"34567c3b2e7101c112115bbafd7b5710e7d212377e56311d8bb00897be55801f"} Oct 30 18:30:44.476928 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.476899 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" event={"ID":"bdb7686e846fff2d163fb7282f45a982","Type":"ContainerStarted","Data":"cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189"} Oct 30 18:30:44.484185 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.484157 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" event={"ID":"09d214f128413268df5e715b75027209","Type":"ContainerStarted","Data":"13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726"} Oct 30 18:30:44.488055 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.487960 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fxrmc" event={"ID":"cf9ba972-af6a-4cda-a59b-f51874d49aae","Type":"ContainerStarted","Data":"dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66"} Oct 30 18:30:44.505538 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.505498 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" event={"ID":"4bb762ff-7409-4218-8421-d0b376a46f98","Type":"ContainerStarted","Data":"77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04"} Oct 30 18:30:44.508988 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:44.508940 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerStarted","Data":"95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d"} Oct 30 18:30:45.015767 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.015678 2584 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:30:45.056176 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.055237 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:45.056176 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.055291 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:45.056176 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.055512 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:45.056176 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.055581 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:30:47.055560929 +0000 UTC m=+5.114679928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:45.056176 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.056080 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:45.056176 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.056134 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret podName:deaa4bf3-e4f9-4b22-9bda-eba0080c0697 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:47.056118761 +0000 UTC m=+5.115237760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret") pod "global-pull-secret-syncer-jw7nd" (UID: "deaa4bf3-e4f9-4b22-9bda-eba0080c0697") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:45.156322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.156281 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:45.156515 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.156452 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:45.156515 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.156471 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:45.156515 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.156484 2584 projected.go:196] Error preparing data for projected volume kube-api-access-j6qqs for pod openshift-network-diagnostics/network-check-target-gwzzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:45.156687 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.156543 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs podName:4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:47.156526279 +0000 UTC m=+5.215645268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6qqs" (UniqueName: "kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs") pod "network-check-target-gwzzt" (UID: "4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:45.397384 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.397295 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:45.397564 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.397441 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:45.397879 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.397860 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:45.397987 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.397968 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:45.398045 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.398032 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:45.398112 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:45.398095 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:45.419540 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.419478 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:25:43 +0000 UTC" deadline="2028-07-02 04:00:31.633043171 +0000 UTC" Oct 30 18:30:45.419540 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:45.419537 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14649h29m46.213509555s" Oct 30 18:30:47.078886 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.075666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:47.078886 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.075726 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:47.078886 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.075898 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:47.078886 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.075955 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:30:51.075940572 +0000 UTC m=+9.135059555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:47.078886 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.076277 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:47.078886 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.076339 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret podName:deaa4bf3-e4f9-4b22-9bda-eba0080c0697 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:51.076323327 +0000 UTC m=+9.135442320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret") pod "global-pull-secret-syncer-jw7nd" (UID: "deaa4bf3-e4f9-4b22-9bda-eba0080c0697") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:47.177584 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.177205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:47.177584 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.177402 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:47.177584 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.177421 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:47.177584 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.177434 2584 projected.go:196] Error preparing data for projected volume kube-api-access-j6qqs for pod openshift-network-diagnostics/network-check-target-gwzzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:47.177584 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.177488 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs podName:4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:51.177471721 +0000 UTC m=+9.236590718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6qqs" (UniqueName: "kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs") pod "network-check-target-gwzzt" (UID: "4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:47.397466 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.397226 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:47.397632 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.397593 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:47.397960 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.397946 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:47.398027 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.398014 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:47.398208 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.398190 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:47.398314 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:47.398273 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:47.563125 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.563081 2584 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="1801c2eb8f7de05caa7bba8022648212b1c611167a2936d555e1416568c3e887" exitCode=0 Oct 30 18:30:47.563261 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:47.563141 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerDied","Data":"1801c2eb8f7de05caa7bba8022648212b1c611167a2936d555e1416568c3e887"} Oct 30 18:30:48.587701 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.587663 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-plcdm" event={"ID":"8b4ec318-a75a-4dd0-9da0-ffc41310b641","Type":"ContainerStarted","Data":"3d6b761cd04849123c0eb79ef555bb8d10532267e64239cd80b786b25f95323d"} Oct 30 18:30:48.601557 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.601051 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" event={"ID":"bdb7686e846fff2d163fb7282f45a982","Type":"ContainerStarted","Data":"f976e0453010ee6679814d94c6a49218e44fa33585f29da70a7c50fe56c0b8b0"} Oct 30 18:30:48.601557 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.601325 2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:30:48.605318 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.605110 2584 generic.go:356] "Generic (PLEG): container finished" podID="09d214f128413268df5e715b75027209" containerID="2ea2aea6331069a08c5e5813d6a3ce9bc720d5b4b518b754c7be824f2a525b2a" exitCode=0 Oct 30 18:30:48.605318 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.605163 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" event={"ID":"09d214f128413268df5e715b75027209","Type":"ContainerDied","Data":"2ea2aea6331069a08c5e5813d6a3ce9bc720d5b4b518b754c7be824f2a525b2a"} Oct 30 18:30:48.624571 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.624506 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal"] Oct 30 18:30:48.624678 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.624662 2584 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:30:48.628741 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.628694 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-plcdm" podStartSLOduration=2.291843547 podStartE2EDuration="6.628678696s" totalImagesPullingTime="4.336835149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:48.628195024 +0000 UTC m=+6.687314031" watchObservedRunningTime="2026-10-30 18:30:48.628678696 +0000 UTC m=+6.687797702" Oct 30 18:30:48.708640 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:48.708596 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" podStartSLOduration=0.708582109 podStartE2EDuration="708.582109ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:48.7077858 +0000 UTC m=+6.766904808" watchObservedRunningTime="2026-10-30 18:30:48.708582109 +0000 UTC m=+6.767701114" Oct 30 18:30:49.309607 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.309573 2584 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:49.310376 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.310355 2584 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:30:49.397986 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.397867 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:49.398056 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:49.397990 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:49.399801 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.398395 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:49.399801 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:49.398497 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:49.399801 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.398552 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:49.399801 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:49.398622 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:49.612244 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.612169 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9nr4w" event={"ID":"11d10c3e-223f-41ef-ac3f-cd428bf43ebd","Type":"ContainerStarted","Data":"cc20f384dd659f38072ecd2078c0fba4cfbc3b767d04faafcfdbf8a04474eb51"} Oct 30 18:30:49.616826 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.616800 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" event={"ID":"09d214f128413268df5e715b75027209","Type":"ContainerStarted","Data":"fb0acbf816f067bf113461e36709c6720894f3c9bbec9d6bf2ff33b586b78375"} Oct 30 18:30:49.652760 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:49.651970 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9nr4w" podStartSLOduration=2.164324241 podStartE2EDuration="7.651952781s" totalImagesPullingTime="5.48762854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:49.651420523 +0000 UTC m=+7.710539531" watchObservedRunningTime="2026-10-30 18:30:49.651952781 +0000 UTC m=+7.711071786" Oct 30 18:30:50.632624 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:50.632591 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" event={"ID":"4bb762ff-7409-4218-8421-d0b376a46f98","Type":"ContainerStarted","Data":"5571a5fee0c117947d28b2f6fd3c2081a079b76683340443149f649121c91115"} Oct 30 18:30:50.633176 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:50.632721 2584 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:30:51.116917 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.116882 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:51.117092 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.116928 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:51.117092 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.117057 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:51.117220 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.117116 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:30:59.117099768 +0000 UTC m=+17.176218752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:51.117625 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.117561 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:51.117625 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.117621 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret podName:deaa4bf3-e4f9-4b22-9bda-eba0080c0697 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:59.117605948 +0000 UTC m=+17.176724934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret") pod "global-pull-secret-syncer-jw7nd" (UID: "deaa4bf3-e4f9-4b22-9bda-eba0080c0697") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:51.217821 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.217722 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:51.217981 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.217909 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:51.217981 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.217929 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:51.217981 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.217941 2584 projected.go:196] Error preparing data for projected volume kube-api-access-j6qqs for pod openshift-network-diagnostics/network-check-target-gwzzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:51.218135 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.217998 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs podName:4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43 nodeName:}" failed. No retries permitted until 2026-10-30 18:30:59.217977888 +0000 UTC m=+17.277096878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6qqs" (UniqueName: "kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs") pod "network-check-target-gwzzt" (UID: "4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:51.397547 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.397510 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:51.397725 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.397630 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:51.397725 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.397713 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:51.397840 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.397798 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:51.397896 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.397855 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:51.397947 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:51.397921 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:51.840971 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:51.840929 2584 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:30:52.404755 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.404656 2584 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:30:51.840950055Z","UUID":"793d7404-1576-4d31-872f-ddad83f0882c","Handler":null,"Name":"","Endpoint":""} Oct 30 18:30:52.406236 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.406214 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:30:52.406236 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.406239 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:30:52.640302 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.640266 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sm8gj" event={"ID":"463c6228-727e-4d4f-9b80-1db615657b4f","Type":"ContainerStarted","Data":"d3c3b97813bde343ec0c3e95aaa40d0fc05b81356e668f96ea678e1871dc64f5"} Oct 30 18:30:52.642011 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.641983 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fxrmc" event={"ID":"cf9ba972-af6a-4cda-a59b-f51874d49aae","Type":"ContainerStarted","Data":"21382e6dc9fdab21b425318552ba42af7dd7285be592b99f12f296123130805e"} Oct 30 18:30:52.644260 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.644231 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" event={"ID":"4bb762ff-7409-4218-8421-d0b376a46f98","Type":"ContainerStarted","Data":"962cdafdefd8e449a213c4526868c3d1741a4da4846d43b1ee265be64002ae58"} Oct 30 18:30:52.646356 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.646334 2584 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="c333f72351781863bdf6926ad8d8992cdd0111623d1c1035a0117858db66e70f" exitCode=0 Oct 30 18:30:52.646455 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.646369 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerDied","Data":"c333f72351781863bdf6926ad8d8992cdd0111623d1c1035a0117858db66e70f"} Oct 30 18:30:52.679165 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.679104 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-sm8gj" podStartSLOduration=1.299112636 podStartE2EDuration="9.679088161s" totalImagesPullingTime="8.379975525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:52.678995599 +0000 UTC m=+10.738114606" watchObservedRunningTime="2026-10-30 18:30:52.679088161 +0000 UTC m=+10.738207165" Oct 30 18:30:52.679306 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:52.679207 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" podStartSLOduration=9.679201131 podStartE2EDuration="9.679201131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:49.691077905 +0000 UTC m=+7.750196915" watchObservedRunningTime="2026-10-30 18:30:52.679201131 +0000 UTC m=+10.738320135" Oct 30 18:30:53.397229 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.396943 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:53.397619 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.396964 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:53.397619 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:53.397334 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:53.397619 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.396970 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:53.397619 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:53.397430 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:53.397619 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:53.397501 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:53.651451 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.650981 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l5zgj" event={"ID":"399ea207-fdb0-4a0a-a841-b113cffc4c3e","Type":"ContainerStarted","Data":"9b8460efe3ae6290dbaf738e9243b13d81c003af4c09030c1c9869d17dc463bb"} Oct 30 18:30:53.653685 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.653648 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" event={"ID":"4bb762ff-7409-4218-8421-d0b376a46f98","Type":"ContainerStarted","Data":"2e988e64b90388498ed5b990254da47d2dd5c15d5be06a9e09cba86a7122cf73"} Oct 30 18:30:53.655988 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.655965 2584 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="7b2aabf97f5284711fdc734d8254d77495295ad980c8a38656512cbc0f793293" exitCode=0 Oct 30 18:30:53.656231 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.656210 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerDied","Data":"7b2aabf97f5284711fdc734d8254d77495295ad980c8a38656512cbc0f793293"} Oct 30 18:30:53.693298 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.693233 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-fxrmc" podStartSLOduration=2.567773213 podStartE2EDuration="10.693220188s" totalImagesPullingTime="8.125446975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:52.764935477 +0000 UTC m=+10.824054481" watchObservedRunningTime="2026-10-30 18:30:53.693220188 +0000 UTC m=+11.752339193" Oct 30 18:30:53.693758 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:53.693600 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l5zgj" podStartSLOduration=3.650199981 podStartE2EDuration="11.693589114s" totalImagesPullingTime="8.043389133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:53.692628729 +0000 UTC m=+11.751747733" watchObservedRunningTime="2026-10-30 18:30:53.693589114 +0000 UTC m=+11.752708120" Oct 30 18:30:54.659826 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:54.659788 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7lgct" event={"ID":"b60126a0-255e-4b6d-9166-18c92777ddc5","Type":"ContainerStarted","Data":"1f509890ff86be5d08e4eb0d5a2b37ec69be9c9337699645abaf32f4f8b6e663"} Oct 30 18:30:54.663946 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:54.663913 2584 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="755070d4d2723437f017a7022b4941c025c1df8c5cbc8cfe501e9764cb2dc8d3" exitCode=0 Oct 30 18:30:54.664085 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:54.663999 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerDied","Data":"755070d4d2723437f017a7022b4941c025c1df8c5cbc8cfe501e9764cb2dc8d3"} Oct 30 18:30:54.702262 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:54.702168 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7lgct" podStartSLOduration=2.256096549 podStartE2EDuration="12.702136037s" totalImagesPullingTime="10.446039488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:54.70156143 +0000 UTC m=+12.760680447" watchObservedRunningTime="2026-10-30 18:30:54.702136037 +0000 UTC m=+12.761255042" Oct 30 18:30:54.702407 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:54.702343 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" podStartSLOduration=2.966933495 podStartE2EDuration="11.7023371s" totalImagesPullingTime="8.735403605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:53.783297422 +0000 UTC m=+11.842416428" watchObservedRunningTime="2026-10-30 18:30:54.7023371 +0000 UTC m=+12.761456107" Oct 30 18:30:55.398175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:55.397548 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:55.398175 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:55.397659 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:55.398175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:55.397717 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:55.398175 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:55.397773 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:55.398175 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:55.397814 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:55.398175 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:55.397858 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:56.672063 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:56.672032 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerStarted","Data":"3bb7caf48f826015d2d712bc2cd31b40b60520b917b27bdd91fac0d7bc79b908"} Oct 30 18:30:57.397544 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:57.397503 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:57.397736 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:57.397560 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:57.397736 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:57.397503 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:57.397736 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:57.397632 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:57.397736 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:57.397700 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:57.397929 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:57.397769 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:57.681262 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:57.681185 2584 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="3bb7caf48f826015d2d712bc2cd31b40b60520b917b27bdd91fac0d7bc79b908" exitCode=0 Oct 30 18:30:57.681262 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:57.681232 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerDied","Data":"3bb7caf48f826015d2d712bc2cd31b40b60520b917b27bdd91fac0d7bc79b908"} Oct 30 18:30:58.686365 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:58.686117 2584 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="778b346007c5482006fc6df4052d835949292287c9f7fd98e59aeb86d29676ba" exitCode=0 Oct 30 18:30:58.686669 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:58.686174 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerDied","Data":"778b346007c5482006fc6df4052d835949292287c9f7fd98e59aeb86d29676ba"} Oct 30 18:30:58.687908 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:58.687889 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"111e447894024b74f5c6265d90877b96409f9e0e2fa25d620a0a572461f9e4e3"} Oct 30 18:30:59.174742 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.174514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:59.174742 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.174699 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:59.174945 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.174653 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:59.174945 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.174784 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:59.174945 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.174814 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret podName:deaa4bf3-e4f9-4b22-9bda-eba0080c0697 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.174797232 +0000 UTC m=+33.233916233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret") pod "global-pull-secret-syncer-jw7nd" (UID: "deaa4bf3-e4f9-4b22-9bda-eba0080c0697") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:30:59.174945 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.174842 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.174835737 +0000 UTC m=+33.233954720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:30:59.275303 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.275276 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:59.275462 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.275391 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:30:59.275462 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.275404 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:30:59.275462 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.275412 2584 projected.go:196] Error preparing data for projected volume kube-api-access-j6qqs for pod openshift-network-diagnostics/network-check-target-gwzzt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:59.275462 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.275455 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs podName:4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.275443095 +0000 UTC m=+33.334562092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j6qqs" (UniqueName: "kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs") pod "network-check-target-gwzzt" (UID: "4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:30:59.397032 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.397001 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:30:59.397190 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.397114 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:30:59.397252 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.397172 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:30:59.397252 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.397189 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:30:59.397311 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.397267 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:30:59.397370 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:30:59.397349 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:30:59.692966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.692944 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:30:59.693407 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.693293 2584 generic.go:356] "Generic (PLEG): container finished" podID="47df06d1-8c08-47c2-9e1f-10636f6bf035" containerID="dbf87f3fac62d960281e4e3abbff71574f47f8e79d8d8350ebd70fb97fac956a" exitCode=1 Oct 30 18:30:59.693407 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.693367 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"61d4bf320f59e2f254e5285928fd48dbffd1148423a41c3552dd25bc8670aa75"} Oct 30 18:30:59.693493 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.693449 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"5c16efa92b75671a5c0d0cd77d8a8b724ec007099a972e88615a903c77e91967"} Oct 30 18:30:59.693529 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.693502 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"223c281dd3938e2dc7abae73af6e46ae27e91d8171c9456e414b5c6797a03a32"} Oct 30 18:30:59.693571 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.693553 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"7bb9fba63ba86e1e1c70c000b5094038d2d77e1da4523a34c184be693edb0a34"} Oct 30 18:30:59.693615 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.693585 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerDied","Data":"dbf87f3fac62d960281e4e3abbff71574f47f8e79d8d8350ebd70fb97fac956a"} Oct 30 18:30:59.696522 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.696500 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" event={"ID":"739cc6ce-77da-492c-9cfc-6bfe5c50dd56","Type":"ContainerStarted","Data":"083fed6fb6ba1962e7e1773bd903f1e78e5566c1e7742919fab10d5034320879"} Oct 30 18:30:59.739945 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:30:59.739898 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jjvbp" podStartSLOduration=7.352420249 podStartE2EDuration="17.739885693s" totalImagesPullingTime="10.387465444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:30:59.739720935 +0000 UTC m=+17.798839940" watchObservedRunningTime="2026-10-30 18:30:59.739885693 +0000 UTC m=+17.799004697" Oct 30 18:31:01.397666 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:01.397583 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:01.397666 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:01.397616 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:01.397666 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:01.397614 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:01.398077 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:01.397686 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:31:01.398077 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:01.397777 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:31:01.398077 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:01.397855 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:31:01.703886 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:01.703811 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:31:01.704191 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:01.704171 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"d4f5e88fc6722155c86fbfc224197b7f8a346a2aad6516931f1efad8f8c04b85"} Oct 30 18:31:03.397823 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.397742 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:03.397823 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.397776 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:03.398517 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:03.397850 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:31:03.398517 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.397925 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:03.398517 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:03.398014 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:31:03.398517 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:03.398089 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:31:03.711869 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.711792 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:31:03.713797 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.713764 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"14fb125117c66054edca4aa6415101c95e003de66917edd9cd7fa19541eeea0d"} Oct 30 18:31:03.714049 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.714015 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:03.714166 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.714099 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:03.714238 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.714181 2584 scope.go:122] "RemoveContainer" containerID="dbf87f3fac62d960281e4e3abbff71574f47f8e79d8d8350ebd70fb97fac956a" Oct 30 18:31:03.729733 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.729713 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:03.729857 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:03.729844 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:04.719569 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:04.719533 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:31:04.719962 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:04.719901 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" event={"ID":"47df06d1-8c08-47c2-9e1f-10636f6bf035","Type":"ContainerStarted","Data":"5d957102517c0be2155ca85a5f2ebfaa612a791dad081516291548be84dbdde3"} Oct 30 18:31:04.720035 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:04.720020 2584 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:04.808387 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:04.808338 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" podStartSLOduration=8.105193557 podStartE2EDuration="22.808325347s" totalImagesPullingTime="14.70313179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:04.808140415 +0000 UTC m=+22.867259419" watchObservedRunningTime="2026-10-30 18:31:04.808325347 +0000 UTC m=+22.867444351" Oct 30 18:31:05.146240 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.146173 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gwzzt"] Oct 30 18:31:05.146358 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.146309 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:05.146399 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:05.146381 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:31:05.150382 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.150359 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlm58"] Oct 30 18:31:05.150484 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.150446 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:05.150532 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:05.150518 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:31:05.150823 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.150805 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jw7nd"] Oct 30 18:31:05.150890 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.150880 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:05.150961 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:05.150946 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:31:05.722122 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:05.722094 2584 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:06.397531 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:06.397503 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:06.397767 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:06.397503 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:06.397767 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:06.397627 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:31:06.397767 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:06.397670 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:31:06.803018 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:06.802792 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:07.397036 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:07.397010 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:07.397226 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:07.397111 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:31:08.397227 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:08.397198 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:08.397684 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:08.397198 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:08.397684 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:08.397293 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:31:08.397684 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:08.397400 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:31:08.549207 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:08.549172 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:08.549365 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:08.549329 2584 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:08.549793 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:08.549772 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:09.396999 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:09.396968 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:09.397169 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:09.397089 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:31:10.397140 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:10.397109 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:10.397581 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:10.397225 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gwzzt" podUID="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" Oct 30 18:31:10.397581 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:10.397266 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:10.397581 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:10.397315 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jw7nd" podUID="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" Oct 30 18:31:11.397803 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.397776 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:11.398246 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:11.397884 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xlm58" podUID="10ed4e5d-002c-48ca-ad1e-199694eef36b" Oct 30 18:31:11.734965 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.734939 2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeReady" Oct 30 18:31:11.735164 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.735070 2584 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:31:11.799081 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.799049 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xzmd9"] Oct 30 18:31:11.843515 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.843489 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-twcqb"] Oct 30 18:31:11.843669 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.843602 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:11.866472 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.866442 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xzmd9"] Oct 30 18:31:11.866600 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.866482 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-twcqb"] Oct 30 18:31:11.866600 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.866549 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:11.866600 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.866579 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dd54f845-4vwhq"] Oct 30 18:31:11.873481 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.873457 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 18:31:11.873578 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.873495 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 18:31:11.882612 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.882591 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-glvc2"] Oct 30 18:31:11.882725 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.882676 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.896812 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.896791 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lpq29"] Oct 30 18:31:11.896948 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.896928 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:11.918777 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.918711 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9q5g7"] Oct 30 18:31:11.918860 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.918836 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:11.929478 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.929440 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:11.929715 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.929699 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 18:31:11.929818 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.929786 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tvrn8\"" Oct 30 18:31:11.929921 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.929822 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 18:31:11.929921 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.929861 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 18:31:11.938697 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.938676 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-zkkkh"] Oct 30 18:31:11.938830 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.938816 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:11.962752 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.962728 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-687ccc8fd4-p2sn5"] Oct 30 18:31:11.962857 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.962833 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:11.963840 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.963818 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.963915 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.963851 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhgb4\" (UniqueName: \"kubernetes.io/projected/a04618ce-f5a4-49ae-8fee-ffb082167b8c-kube-api-access-xhgb4\") pod \"kas-connection-checker-685895c7c9-xzmd9\" (UID: \"a04618ce-f5a4-49ae-8fee-ffb082167b8c\") " pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:11.963915 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.963871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.963915 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.963891 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.964009 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.963949 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.964055 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.964037 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.964137 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.964122 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.964256 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.964239 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcsbm\" (UniqueName: \"kubernetes.io/projected/9c695b41-2900-4b0d-832b-bb35456eb816-kube-api-access-mcsbm\") pod \"kas-connection-checker-685895c7c9-twcqb\" (UID: \"9c695b41-2900-4b0d-832b-bb35456eb816\") " pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:11.964292 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.964276 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.964323 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.964302 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:11.974279 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.974260 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-75fdfd5b99-vpjd7"] Oct 30 18:31:11.974384 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.974371 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:11.979561 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.979545 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 18:31:11.979922 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.979907 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:31:11.990376 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.990335 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc"] Oct 30 18:31:11.990449 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.990430 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:11.993868 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:11.993847 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 18:31:12.002771 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.002756 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9"] Oct 30 18:31:12.002888 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.002873 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.012831 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.012815 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n922x"] Oct 30 18:31:12.012925 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.012912 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.024200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.024182 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8"] Oct 30 18:31:12.024293 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.024279 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.029257 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029243 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 18:31:12.029321 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029276 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 18:31:12.029321 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029276 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 18:31:12.029433 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029372 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:31:12.029489 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029446 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtqtd\"" Oct 30 18:31:12.029489 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029451 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 18:31:12.029489 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029468 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-bqvz6\"" Oct 30 18:31:12.029489 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029479 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 18:31:12.029708 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.029687 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 18:31:12.030056 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.030042 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 18:31:12.036557 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.036541 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-c262b"] Oct 30 18:31:12.036643 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.036618 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.046701 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.046686 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz"] Oct 30 18:31:12.046789 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.046777 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.058965 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.058949 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jr9ld"] Oct 30 18:31:12.059075 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.059061 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:12.064600 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064581 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-stats-auth\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.064694 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064616 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.064694 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064637 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79d5b339-6665-4fbe-92db-77fd202119d2-tmp-dir\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.064694 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064673 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.064855 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064743 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpkgn\" (UniqueName: \"kubernetes.io/projected/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-kube-api-access-zpkgn\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.064855 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064775 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcsbm\" (UniqueName: \"kubernetes.io/projected/9c695b41-2900-4b0d-832b-bb35456eb816-kube-api-access-mcsbm\") pod \"kas-connection-checker-685895c7c9-twcqb\" (UID: \"9c695b41-2900-4b0d-832b-bb35456eb816\") " pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:12.064855 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064803 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:12.064855 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064831 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.065054 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064864 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.065054 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.064984 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065054 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065010 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l8qg\" (UniqueName: \"kubernetes.io/projected/13093b2f-a782-4521-b80f-f4ee194af653-kube-api-access-5l8qg\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.065054 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065039 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2fdf2c77-42d8-4686-a656-334557d55531-nginx-conf\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:12.065253 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065118 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68rjt\" (UniqueName: \"kubernetes.io/projected/0fae4e48-2851-4c67-9615-ad86f9795397-kube-api-access-68rjt\") pod \"kas-connection-checker-685895c7c9-9q5g7\" (UID: \"0fae4e48-2851-4c67-9615-ad86f9795397\") " pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:12.065253 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065179 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-service-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.065253 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065230 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065397 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065308 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.065397 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065339 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.065397 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065369 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.065542 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065397 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-default-certificate\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.065542 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065421 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13093b2f-a782-4521-b80f-f4ee194af653-serving-cert\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.065542 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065449 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065542 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065489 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.065739 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065563 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065739 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065604 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065739 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065631 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065739 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065721 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-tmp\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.065895 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065748 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.065895 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065785 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.065895 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065840 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.066038 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.066038 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.065972 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.066038 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066003 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.066038 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066005 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79d5b339-6665-4fbe-92db-77fd202119d2-config-volume\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.066227 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066068 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.066227 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4m7\" (UniqueName: \"kubernetes.io/projected/79d5b339-6665-4fbe-92db-77fd202119d2-kube-api-access-tw4m7\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.066227 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066136 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-snapshots\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.066227 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066193 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhgb4\" (UniqueName: \"kubernetes.io/projected/a04618ce-f5a4-49ae-8fee-ffb082167b8c-kube-api-access-xhgb4\") pod \"kas-connection-checker-685895c7c9-xzmd9\" (UID: \"a04618ce-f5a4-49ae-8fee-ffb082167b8c\") " pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:12.066227 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.066219 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.066423 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.066383 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:12.066423 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.066395 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd54f845-4vwhq: secret "image-registry-tls" not found Oct 30 18:31:12.066483 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.066433 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls podName:865d20d3-b981-4d85-a56b-4951ae770312 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.566420017 +0000 UTC m=+30.625539000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls") pod "image-registry-6dd54f845-4vwhq" (UID: "865d20d3-b981-4d85-a56b-4951ae770312") : secret "image-registry-tls" not found Oct 30 18:31:12.067182 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.067161 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.068647 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.068627 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.068719 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.068636 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.071735 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.071719 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf"] Oct 30 18:31:12.071824 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.071811 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.075953 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.075933 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.076711 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.076690 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 18:31:12.076787 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.076730 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 18:31:12.076787 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.076737 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.076787 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.076738 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.077019 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077003 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.077019 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077014 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077176 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077018 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077176 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077038 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.077176 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077064 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.077314 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077199 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077314 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077211 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.077314 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077228 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 18:31:12.077492 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077475 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077492 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077488 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077627 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077507 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 18:31:12.077627 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077516 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077627 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077523 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.077627 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077526 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 18:31:12.077627 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077534 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 18:31:12.077627 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077545 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.077798 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077762 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.078009 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.077994 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 18:31:12.082224 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082205 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9q5g7"] Oct 30 18:31:12.082313 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082243 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n922x"] Oct 30 18:31:12.082313 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082265 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc"] Oct 30 18:31:12.082313 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082290 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8"] Oct 30 18:31:12.082313 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082309 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75fdfd5b99-vpjd7"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082313 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082327 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-glvc2"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082347 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-zkkkh"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082365 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd54f845-4vwhq"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082383 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082401 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lpq29"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082426 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jr9ld"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082445 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-c262b"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082470 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082490 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687ccc8fd4-p2sn5"] Oct 30 18:31:12.082535 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.082507 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9"] Oct 30 18:31:12.091789 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.091769 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 18:31:12.092896 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.092875 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 18:31:12.094236 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.094219 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcsbm\" (UniqueName: \"kubernetes.io/projected/9c695b41-2900-4b0d-832b-bb35456eb816-kube-api-access-mcsbm\") pod \"kas-connection-checker-685895c7c9-twcqb\" (UID: \"9c695b41-2900-4b0d-832b-bb35456eb816\") " pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:12.094819 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.094795 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.094890 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.094861 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.097431 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.097410 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhgb4\" (UniqueName: \"kubernetes.io/projected/a04618ce-f5a4-49ae-8fee-ffb082167b8c-kube-api-access-xhgb4\") pod \"kas-connection-checker-685895c7c9-xzmd9\" (UID: \"a04618ce-f5a4-49ae-8fee-ffb082167b8c\") " pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:12.129485 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.129459 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 18:31:12.129485 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.129463 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 18:31:12.129725 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.129710 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 18:31:12.129795 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.129726 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:31:12.129795 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.129752 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wrnjz\"" Oct 30 18:31:12.129795 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.129756 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-shvwg\"" Oct 30 18:31:12.167039 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.167132 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167047 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5scv\" (UniqueName: \"kubernetes.io/projected/9f0e7740-17a0-460e-9407-5650fa708046-kube-api-access-h5scv\") pod \"volume-data-source-validator-58c9f5f46d-c4nbz\" (UID: \"9f0e7740-17a0-460e-9407-5650fa708046\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:12.167132 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167064 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a3d167-dbc7-4155-aee3-e056b558ab8a-cert\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.167132 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167082 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.167132 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167097 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79d5b339-6665-4fbe-92db-77fd202119d2-config-volume\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.167319 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167136 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4m7\" (UniqueName: \"kubernetes.io/projected/79d5b339-6665-4fbe-92db-77fd202119d2-kube-api-access-tw4m7\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.167319 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167184 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-snapshots\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.167319 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.167218 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:12.167319 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167228 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwfcw\" (UniqueName: \"kubernetes.io/projected/30e186ca-854c-417a-96ed-3a660db8b8f4-kube-api-access-xwfcw\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.167319 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.167260 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.667244901 +0000 UTC m=+30.726363884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:12.167319 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167277 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-stats-auth\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.167605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167457 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.167605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167485 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdzm6\" (UniqueName: \"kubernetes.io/projected/06297482-f53e-497f-8012-1c6a55b1462b-kube-api-access-qdzm6\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.167605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167503 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-config\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.167605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167524 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.167605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167552 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79d5b339-6665-4fbe-92db-77fd202119d2-tmp-dir\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.167605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167515 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167609 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167677 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzw9c\" (UniqueName: \"kubernetes.io/projected/56a08134-6094-4349-b252-ccf6d0aa8774-kube-api-access-bzw9c\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.167712 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.167765 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.667749161 +0000 UTC m=+30.726868149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167768 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79d5b339-6665-4fbe-92db-77fd202119d2-tmp-dir\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167785 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.167809 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.667797669 +0000 UTC m=+30.726916658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : configmap references non-existent config key: service-ca.crt Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167829 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frs5z\" (UniqueName: \"kubernetes.io/projected/68a3d167-dbc7-4155-aee3-e056b558ab8a-kube-api-access-frs5z\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167856 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/06297482-f53e-497f-8012-1c6a55b1462b-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167884 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpkgn\" (UniqueName: \"kubernetes.io/projected/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-kube-api-access-zpkgn\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.167905 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167911 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30e186ca-854c-417a-96ed-3a660db8b8f4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167972 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-serving-cert\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.167977 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79d5b339-6665-4fbe-92db-77fd202119d2-config-volume\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168034 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168101 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652nf\" (UniqueName: \"kubernetes.io/projected/72c54cf4-7294-4d5b-a0f5-61d989cd960f-kube-api-access-652nf\") pod \"network-check-source-6f78d59f99-xjnjf\" (UID: \"72c54cf4-7294-4d5b-a0f5-61d989cd960f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.168132 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168132 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-serving-cert\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.168194 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.668179538 +0000 UTC m=+30.727298525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-trusted-ca\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168273 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168313 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8qg\" (UniqueName: \"kubernetes.io/projected/13093b2f-a782-4521-b80f-f4ee194af653-kube-api-access-5l8qg\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.168328 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168324 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-snapshots\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168480 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2fdf2c77-42d8-4686-a656-334557d55531-nginx-conf\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68rjt\" (UniqueName: \"kubernetes.io/projected/0fae4e48-2851-4c67-9615-ad86f9795397-kube-api-access-68rjt\") pod \"kas-connection-checker-685895c7c9-9q5g7\" (UID: \"0fae4e48-2851-4c67-9615-ad86f9795397\") " pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168545 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-config\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168577 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-service-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168641 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-default-certificate\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168665 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13093b2f-a782-4521-b80f-f4ee194af653-serving-cert\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168692 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168747 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168772 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30e186ca-854c-417a-96ed-3a660db8b8f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168811 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168840 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168858 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.168894 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.168865 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt78g\" (UniqueName: \"kubernetes.io/projected/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-kube-api-access-zt78g\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.169564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169000 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2fdf2c77-42d8-4686-a656-334557d55531-nginx-conf\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:12.169564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169386 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-service-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.169706 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169680 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.169766 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169702 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-stats-auth\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.169766 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169746 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcbsv\" (UniqueName: \"kubernetes.io/projected/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-kube-api-access-dcbsv\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.169868 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169804 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-tmp\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.169921 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169832 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.169975 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.169923 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.170122 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.170074 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.170218 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.170194 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-tmp\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.170313 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.170298 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:12.170359 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.170315 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687ccc8fd4-p2sn5: secret "image-registry-tls" not found Oct 30 18:31:12.170411 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.170396 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls podName:9e6926c1-1692-4111-85b6-319433b9bdfe nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.670379432 +0000 UTC m=+30.729498433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls") pod "image-registry-687ccc8fd4-p2sn5" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe") : secret "image-registry-tls" not found Oct 30 18:31:12.171529 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.171512 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-default-certificate\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.171658 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.171638 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13093b2f-a782-4521-b80f-f4ee194af653-serving-cert\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.171732 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.171711 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.172249 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.172232 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.176206 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.176186 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.176302 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.176225 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:31:12.176302 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.176279 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.176393 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.176304 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:31:12.176670 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.176656 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:12.191387 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.191369 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:12.196161 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.196122 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.196249 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.196183 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8qg\" (UniqueName: \"kubernetes.io/projected/13093b2f-a782-4521-b80f-f4ee194af653-kube-api-access-5l8qg\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.197419 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.197396 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4m7\" (UniqueName: \"kubernetes.io/projected/79d5b339-6665-4fbe-92db-77fd202119d2-kube-api-access-tw4m7\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.197970 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.197954 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.198486 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.198467 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpkgn\" (UniqueName: \"kubernetes.io/projected/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-kube-api-access-zpkgn\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.199879 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.199864 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68rjt\" (UniqueName: \"kubernetes.io/projected/0fae4e48-2851-4c67-9615-ad86f9795397-kube-api-access-68rjt\") pod \"kas-connection-checker-685895c7c9-9q5g7\" (UID: \"0fae4e48-2851-4c67-9615-ad86f9795397\") " pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:12.217248 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.217231 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:12.270420 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270385 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-config\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.270564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270454 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30e186ca-854c-417a-96ed-3a660db8b8f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.270564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270483 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt78g\" (UniqueName: \"kubernetes.io/projected/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-kube-api-access-zt78g\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.270564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270503 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcbsv\" (UniqueName: \"kubernetes.io/projected/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-kube-api-access-dcbsv\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.270564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.270564 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270549 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5scv\" (UniqueName: \"kubernetes.io/projected/9f0e7740-17a0-460e-9407-5650fa708046-kube-api-access-h5scv\") pod \"volume-data-source-validator-58c9f5f46d-c4nbz\" (UID: \"9f0e7740-17a0-460e-9407-5650fa708046\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:12.270860 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270843 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a3d167-dbc7-4155-aee3-e056b558ab8a-cert\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.270937 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.270896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfcw\" (UniqueName: \"kubernetes.io/projected/30e186ca-854c-417a-96ed-3a660db8b8f4-kube-api-access-xwfcw\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.271028 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271006 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-config\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.271088 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.271157 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdzm6\" (UniqueName: \"kubernetes.io/projected/06297482-f53e-497f-8012-1c6a55b1462b-kube-api-access-qdzm6\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.271157 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.271122 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:12.271157 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271135 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-config\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.271310 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.271205 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.771186073 +0000 UTC m=+30.830305059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:12.271310 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271229 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzw9c\" (UniqueName: \"kubernetes.io/projected/56a08134-6094-4349-b252-ccf6d0aa8774-kube-api-access-bzw9c\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.271310 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271264 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.271461 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.271339 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:12.271461 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271269 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.271461 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.271380 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:12.771367224 +0000 UTC m=+30.830486213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:12.271461 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271398 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frs5z\" (UniqueName: \"kubernetes.io/projected/68a3d167-dbc7-4155-aee3-e056b558ab8a-kube-api-access-frs5z\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.271461 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271428 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/06297482-f53e-497f-8012-1c6a55b1462b-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.271461 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271460 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30e186ca-854c-417a-96ed-3a660db8b8f4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.271755 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271486 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-serving-cert\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.271755 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271526 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652nf\" (UniqueName: \"kubernetes.io/projected/72c54cf4-7294-4d5b-a0f5-61d989cd960f-kube-api-access-652nf\") pod \"network-check-source-6f78d59f99-xjnjf\" (UID: \"72c54cf4-7294-4d5b-a0f5-61d989cd960f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:12.271755 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271565 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-serving-cert\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.271755 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.271611 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-trusted-ca\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.272038 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.272012 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-config\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.272096 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.272053 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30e186ca-854c-417a-96ed-3a660db8b8f4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.272284 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.272260 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/06297482-f53e-497f-8012-1c6a55b1462b-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.272449 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.272424 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-trusted-ca\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.273463 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.273418 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30e186ca-854c-417a-96ed-3a660db8b8f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.274293 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.274270 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-serving-cert\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.274509 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.274493 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a3d167-dbc7-4155-aee3-e056b558ab8a-cert\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.274875 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.274861 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-serving-cert\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.286623 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.286604 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:12.295447 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.295422 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:12.298379 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.298360 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcbsv\" (UniqueName: \"kubernetes.io/projected/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-kube-api-access-dcbsv\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.298852 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.298801 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5scv\" (UniqueName: \"kubernetes.io/projected/9f0e7740-17a0-460e-9407-5650fa708046-kube-api-access-h5scv\") pod \"volume-data-source-validator-58c9f5f46d-c4nbz\" (UID: \"9f0e7740-17a0-460e-9407-5650fa708046\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:12.300327 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.300287 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt78g\" (UniqueName: \"kubernetes.io/projected/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-kube-api-access-zt78g\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.308698 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.308671 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzw9c\" (UniqueName: \"kubernetes.io/projected/56a08134-6094-4349-b252-ccf6d0aa8774-kube-api-access-bzw9c\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.310466 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.309524 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdzm6\" (UniqueName: \"kubernetes.io/projected/06297482-f53e-497f-8012-1c6a55b1462b-kube-api-access-qdzm6\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.310582 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.310539 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-652nf\" (UniqueName: \"kubernetes.io/projected/72c54cf4-7294-4d5b-a0f5-61d989cd960f-kube-api-access-652nf\") pod \"network-check-source-6f78d59f99-xjnjf\" (UID: \"72c54cf4-7294-4d5b-a0f5-61d989cd960f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:12.313202 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.311065 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwfcw\" (UniqueName: \"kubernetes.io/projected/30e186ca-854c-417a-96ed-3a660db8b8f4-kube-api-access-xwfcw\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.320487 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.320428 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frs5z\" (UniqueName: \"kubernetes.io/projected/68a3d167-dbc7-4155-aee3-e056b558ab8a-kube-api-access-frs5z\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.355720 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.355692 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:12.365397 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.365369 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:12.376229 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.376210 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:12.388443 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.388333 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:12.400026 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.397369 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:12.400026 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.397842 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:12.414305 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.414224 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:31:12.414305 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.414227 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:31:12.414433 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.414311 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:31:12.423186 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.423102 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:12.437942 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.437918 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:12.560022 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.559995 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-twcqb"] Oct 30 18:31:12.560107 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.560096 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xzmd9"] Oct 30 18:31:12.574259 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.574232 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:12.574391 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.574373 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:12.574427 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.574393 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd54f845-4vwhq: secret "image-registry-tls" not found Oct 30 18:31:12.574459 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.574445 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls podName:865d20d3-b981-4d85-a56b-4951ae770312 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.574431422 +0000 UTC m=+31.633550431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls") pod "image-registry-6dd54f845-4vwhq" (UID: "865d20d3-b981-4d85-a56b-4951ae770312") : secret "image-registry-tls" not found Oct 30 18:31:12.633279 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.633194 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-9q5g7"] Oct 30 18:31:12.638370 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.638334 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fae4e48_2851_4c67_9615_ad86f9795397.slice/crio-44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1 WatchSource:0}: Error finding container 44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1: Status 404 returned error can't find the container with id 44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1 Oct 30 18:31:12.668897 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.668829 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-zkkkh"] Oct 30 18:31:12.674793 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.674763 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:12.674891 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.674845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.674946 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.674906 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:12.674946 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.674935 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:12.675043 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.674990 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:12.675181 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675164 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:12.675250 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675229 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.67521065 +0000 UTC m=+31.734329634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:12.675306 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675273 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:12.675352 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675331 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.675313354 +0000 UTC m=+31.734432338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:12.675403 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.675369 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13093b2f_a782_4521_b80f_f4ee194af653.slice/crio-6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211 WatchSource:0}: Error finding container 6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211: Status 404 returned error can't find the container with id 6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211 Oct 30 18:31:12.675503 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675431 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:12.675503 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675442 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687ccc8fd4-p2sn5: secret "image-registry-tls" not found Oct 30 18:31:12.675503 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675489 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls podName:9e6926c1-1692-4111-85b6-319433b9bdfe nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.675478048 +0000 UTC m=+31.734597031 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls") pod "image-registry-687ccc8fd4-p2sn5" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe") : secret "image-registry-tls" not found Oct 30 18:31:12.675654 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675570 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.675559711 +0000 UTC m=+31.734678696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : configmap references non-existent config key: service-ca.crt Oct 30 18:31:12.675704 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675674 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:12.675957 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.675939 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.675927286 +0000 UTC m=+31.735046269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:12.682918 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.682270 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9"] Oct 30 18:31:12.685078 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.684313 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c4a917_c2ba_46e3_8b7f_d528b1c26bc9.slice/crio-272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf WatchSource:0}: Error finding container 272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf: Status 404 returned error can't find the container with id 272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf Oct 30 18:31:12.727694 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.717079 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8"] Oct 30 18:31:12.739100 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.739034 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" event={"ID":"a04618ce-f5a4-49ae-8fee-ffb082167b8c","Type":"ContainerStarted","Data":"40655e2cdd4d2c280ec2b728030e86b7e2cbc2f7b9e6a7a4eb0ae15d0d1b9cef"} Oct 30 18:31:12.740518 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.740489 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" event={"ID":"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9","Type":"ContainerStarted","Data":"272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf"} Oct 30 18:31:12.741567 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.741546 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" event={"ID":"0fae4e48-2851-4c67-9615-ad86f9795397","Type":"ContainerStarted","Data":"44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1"} Oct 30 18:31:12.742881 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.742860 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" event={"ID":"13093b2f-a782-4521-b80f-f4ee194af653","Type":"ContainerStarted","Data":"6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211"} Oct 30 18:31:12.743919 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.743901 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-twcqb" event={"ID":"9c695b41-2900-4b0d-832b-bb35456eb816","Type":"ContainerStarted","Data":"e9ec10695ea7fed16e26429e0a0845446487a7e50122d5919342abac635526a1"} Oct 30 18:31:12.751820 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.751690 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-c262b"] Oct 30 18:31:12.754197 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.754172 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb14d800_4d5f_4a7c_bade_81c0f4f5c3f3.slice/crio-4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563 WatchSource:0}: Error finding container 4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563: Status 404 returned error can't find the container with id 4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563 Oct 30 18:31:12.776383 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.776360 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:12.776472 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.776411 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:12.776518 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.776495 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:12.776556 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.776533 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:12.776556 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.776545 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.776530635 +0000 UTC m=+31.835649637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:12.776624 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:12.776569 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:13.776558075 +0000 UTC m=+31.835677059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:12.791257 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.791237 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz"] Oct 30 18:31:12.793329 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.793304 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0e7740_17a0_460e_9407_5650fa708046.slice/crio-46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc WatchSource:0}: Error finding container 46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc: Status 404 returned error can't find the container with id 46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc Oct 30 18:31:12.796941 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.796917 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf"] Oct 30 18:31:12.798745 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.798728 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72c54cf4_7294_4d5b_a0f5_61d989cd960f.slice/crio-a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9 WatchSource:0}: Error finding container a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9: Status 404 returned error can't find the container with id a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9 Oct 30 18:31:12.830299 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:12.830279 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jr9ld"] Oct 30 18:31:12.832217 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:12.832193 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a3d167_dbc7_4155_aee3_e056b558ab8a.slice/crio-55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0 WatchSource:0}: Error finding container 55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0: Status 404 returned error can't find the container with id 55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0 Oct 30 18:31:13.396955 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.396920 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:13.414039 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.414012 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:31:13.414517 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.414281 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:31:13.585438 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.584510 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:13.585438 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.584666 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:13.585438 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.584683 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd54f845-4vwhq: secret "image-registry-tls" not found Oct 30 18:31:13.585438 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.584742 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls podName:865d20d3-b981-4d85-a56b-4951ae770312 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.584722116 +0000 UTC m=+33.643841104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls") pod "image-registry-6dd54f845-4vwhq" (UID: "865d20d3-b981-4d85-a56b-4951ae770312") : secret "image-registry-tls" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.685496 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.685584 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.685647 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.685674 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.685728 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.685894 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.685960 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.685940126 +0000 UTC m=+33.745059115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686365 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686379 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687ccc8fd4-p2sn5: secret "image-registry-tls" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686421 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls podName:9e6926c1-1692-4111-85b6-319433b9bdfe nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.686407374 +0000 UTC m=+33.745526373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls") pod "image-registry-687ccc8fd4-p2sn5" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe") : secret "image-registry-tls" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686478 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686510 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.686500226 +0000 UTC m=+33.745619213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686557 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686589 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.686579613 +0000 UTC m=+33.745698598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:13.686768 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.686647 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.686638356 +0000 UTC m=+33.745757342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : configmap references non-existent config key: service-ca.crt Oct 30 18:31:13.754948 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.754914 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" event={"ID":"a04618ce-f5a4-49ae-8fee-ffb082167b8c","Type":"ContainerStarted","Data":"9ea2f97632f91457c2a1251d55bffed3849194987bba97ae88a5435601d16949"} Oct 30 18:31:13.758291 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.758213 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jr9ld" event={"ID":"68a3d167-dbc7-4155-aee3-e056b558ab8a","Type":"ContainerStarted","Data":"55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0"} Oct 30 18:31:13.760313 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.760288 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" event={"ID":"72c54cf4-7294-4d5b-a0f5-61d989cd960f","Type":"ContainerStarted","Data":"a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9"} Oct 30 18:31:13.762036 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.762012 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" event={"ID":"9f0e7740-17a0-460e-9407-5650fa708046","Type":"ContainerStarted","Data":"46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc"} Oct 30 18:31:13.764563 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.764540 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" event={"ID":"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3","Type":"ContainerStarted","Data":"4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563"} Oct 30 18:31:13.768798 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.768773 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" event={"ID":"0fae4e48-2851-4c67-9615-ad86f9795397","Type":"ContainerStarted","Data":"e0a633794200310a1b19c52bade767f88d3358d0099c945f8d2d779a8e1fbab6"} Oct 30 18:31:13.772260 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.771741 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-twcqb" event={"ID":"9c695b41-2900-4b0d-832b-bb35456eb816","Type":"ContainerStarted","Data":"9ef21da33a91a04b98e07b26ccb1991d63f64d98f8d66c8f3845264b705c8520"} Oct 30 18:31:13.774954 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.774910 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" event={"ID":"30e186ca-854c-417a-96ed-3a660db8b8f4","Type":"ContainerStarted","Data":"c11638a51ea397d7f9f3bddbd6ad0601fc716e232571f4e369d7c0b8d107fce8"} Oct 30 18:31:13.786670 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.786644 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:13.786753 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.786718 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:13.786853 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.786834 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:13.786908 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.786885 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:13.786908 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.786901 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.786883202 +0000 UTC m=+33.846002191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:13.787022 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:13.786940 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:15.786923446 +0000 UTC m=+33.846042446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:13.799179 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.797159 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" podStartSLOduration=231.797133293 podStartE2EDuration="3m51.797133293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:13.796688909 +0000 UTC m=+31.855807915" watchObservedRunningTime="2026-10-30 18:31:13.797133293 +0000 UTC m=+31.856252298" Oct 30 18:31:13.884829 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.884476 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-twcqb" podStartSLOduration=231.88446075 podStartE2EDuration="3m51.88446075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:13.838991686 +0000 UTC m=+31.898110702" watchObservedRunningTime="2026-10-30 18:31:13.88446075 +0000 UTC m=+31.943579788" Oct 30 18:31:13.963930 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:13.963822 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:14.124923 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:14.124869 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:15.203324 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.203285 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:15.203787 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.203337 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:15.203787 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.203610 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 18:31:15.203787 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.203679 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs podName:10ed4e5d-002c-48ca-ad1e-199694eef36b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:47.203658197 +0000 UTC m=+65.262777183 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs") pod "network-metrics-daemon-xlm58" (UID: "10ed4e5d-002c-48ca-ad1e-199694eef36b") : secret "metrics-daemon-secret" not found Oct 30 18:31:15.224596 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.224566 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:15.304322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.304278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:15.311528 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.311508 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:15.430401 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.430178 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:15.463010 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.462983 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:15.607843 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.607481 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:15.607843 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.607773 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:15.607843 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.607790 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd54f845-4vwhq: secret "image-registry-tls" not found Oct 30 18:31:15.607843 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.607848 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls podName:865d20d3-b981-4d85-a56b-4951ae770312 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.607830336 +0000 UTC m=+37.666949325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls") pod "image-registry-6dd54f845-4vwhq" (UID: "865d20d3-b981-4d85-a56b-4951ae770312") : secret "image-registry-tls" not found Oct 30 18:31:15.708630 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.708547 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:15.708775 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.708653 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:15.708775 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.708707 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:15.708775 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.708767 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:15.708936 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.708794 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:15.708990 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.708960 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.708942235 +0000 UTC m=+37.768061219 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : configmap references non-existent config key: service-ca.crt Oct 30 18:31:15.709426 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709405 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:15.709523 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709469 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.709452073 +0000 UTC m=+37.768571073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:15.709588 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709537 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:15.709588 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709548 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687ccc8fd4-p2sn5: secret "image-registry-tls" not found Oct 30 18:31:15.709588 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709580 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls podName:9e6926c1-1692-4111-85b6-319433b9bdfe nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.709569086 +0000 UTC m=+37.768688086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls") pod "image-registry-687ccc8fd4-p2sn5" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe") : secret "image-registry-tls" not found Oct 30 18:31:15.709730 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709638 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:15.709730 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709668 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.709659055 +0000 UTC m=+37.768778057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:15.709730 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709724 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:15.709867 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.709751 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.709742139 +0000 UTC m=+37.768861123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:15.749165 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.749095 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" podStartSLOduration=233.749076388 podStartE2EDuration="3m53.749076388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:13.885051793 +0000 UTC m=+31.944170798" watchObservedRunningTime="2026-10-30 18:31:15.749076388 +0000 UTC m=+33.808195394" Oct 30 18:31:15.750181 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.750155 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jw7nd"] Oct 30 18:31:15.754758 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:15.754727 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeaa4bf3_e4f9_4b22_9bda_eba0080c0697.slice/crio-2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1 WatchSource:0}: Error finding container 2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1: Status 404 returned error can't find the container with id 2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1 Oct 30 18:31:15.792610 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.792572 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" event={"ID":"9f0e7740-17a0-460e-9407-5650fa708046","Type":"ContainerStarted","Data":"071daa81aa9d5d5cdb55428820d7eebdbfb268a24eeb2634e664c77886d65852"} Oct 30 18:31:15.796477 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.796443 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jw7nd" event={"ID":"deaa4bf3-e4f9-4b22-9bda-eba0080c0697","Type":"ContainerStarted","Data":"2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1"} Oct 30 18:31:15.809583 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.809547 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:15.809762 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.809627 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:15.809762 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.809726 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:15.809762 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.809752 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:15.809913 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.809795 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.809774959 +0000 UTC m=+37.868893958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:15.809913 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:15.809815 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:19.809805673 +0000 UTC m=+37.868924660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:15.826479 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.826437 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gwzzt"] Oct 30 18:31:15.830482 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:15.830453 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1513cf_a8c5_478a_9cb9_2e99f2f2ef43.slice/crio-d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c WatchSource:0}: Error finding container d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c: Status 404 returned error can't find the container with id d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c Oct 30 18:31:15.839978 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:15.838688 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" podStartSLOduration=215.369569798 podStartE2EDuration="3m37.83867474s" totalImagesPullingTime="2.469104942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:15.838628781 +0000 UTC m=+33.897747790" watchObservedRunningTime="2026-10-30 18:31:15.83867474 +0000 UTC m=+33.897793772" Oct 30 18:31:16.065094 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:16.064953 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:16.228292 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:16.228204 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:16.803847 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:16.803816 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gwzzt" event={"ID":"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43","Type":"ContainerStarted","Data":"d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c"} Oct 30 18:31:18.824976 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.824894 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" event={"ID":"13093b2f-a782-4521-b80f-f4ee194af653","Type":"ContainerStarted","Data":"b4fc2b96770a2989ad974c312aafe702811fe06b2883a953f3b085a6af6560ce"} Oct 30 18:31:18.827764 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.827731 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" event={"ID":"30e186ca-854c-417a-96ed-3a660db8b8f4","Type":"ContainerStarted","Data":"0410e4b9880b8c4646d68d19f8319c518a17a6423241214e10ed01509a47ded4"} Oct 30 18:31:18.830787 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.830756 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jr9ld" event={"ID":"68a3d167-dbc7-4155-aee3-e056b558ab8a","Type":"ContainerStarted","Data":"4558ff8c261feffe9a7ec2da285bbc43475432a7298e7f52763327ac1bd886e8"} Oct 30 18:31:18.832907 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.832885 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/0.log" Oct 30 18:31:18.833008 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.832925 2584 generic.go:356] "Generic (PLEG): container finished" podID="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" containerID="155fe592ba0cc001a90293f399f2f33469c7eb8970cdf05bee2ce36800b4394f" exitCode=255 Oct 30 18:31:18.833065 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.833013 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" event={"ID":"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3","Type":"ContainerDied","Data":"155fe592ba0cc001a90293f399f2f33469c7eb8970cdf05bee2ce36800b4394f"} Oct 30 18:31:18.833188 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.833171 2584 scope.go:122] "RemoveContainer" containerID="155fe592ba0cc001a90293f399f2f33469c7eb8970cdf05bee2ce36800b4394f" Oct 30 18:31:18.835010 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.834577 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" event={"ID":"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9","Type":"ContainerStarted","Data":"2abe84ba6e4f91b36288785de2ba877a454d2a7d6682c1d57dbda7cfbc546199"} Oct 30 18:31:18.870215 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.870134 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" podStartSLOduration=222.267937262 podStartE2EDuration="3m47.870113643s" totalImagesPullingTime="5.602176381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:18.869433624 +0000 UTC m=+36.928552655" watchObservedRunningTime="2026-10-30 18:31:18.870113643 +0000 UTC m=+36.929232649" Oct 30 18:31:18.913064 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:18.913012 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" podStartSLOduration=217.318093117 podStartE2EDuration="3m42.912995034s" totalImagesPullingTime="5.594901917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:18.912085161 +0000 UTC m=+36.971204191" watchObservedRunningTime="2026-10-30 18:31:18.912995034 +0000 UTC m=+36.972114039" Oct 30 18:31:19.002501 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.002440 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" podStartSLOduration=216.428920285 podStartE2EDuration="3m42.002419498s" totalImagesPullingTime="5.573499213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:19.001111404 +0000 UTC m=+37.060230433" watchObservedRunningTime="2026-10-30 18:31:19.002419498 +0000 UTC m=+37.061538504" Oct 30 18:31:19.054447 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.054388 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jr9ld" podStartSLOduration=2.605324176 podStartE2EDuration="8.054370455s" totalImagesPullingTime="5.449046279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:19.053400989 +0000 UTC m=+37.112520021" watchObservedRunningTime="2026-10-30 18:31:19.054370455 +0000 UTC m=+37.113489463" Oct 30 18:31:19.485477 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.484305 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5"] Oct 30 18:31:19.493002 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.492967 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:19.501123 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.501064 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc"] Oct 30 18:31:19.501339 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.501204 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:19.519349 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.519308 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5"] Oct 30 18:31:19.519566 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.519508 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:19.521033 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.521009 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc"] Oct 30 18:31:19.531363 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.531192 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:19.531363 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.531331 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 18:31:19.586966 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.586939 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pvt2z\"" Oct 30 18:31:19.636119 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.636094 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-srvzm"] Oct 30 18:31:19.659706 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.659677 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.660188 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.660097 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:19.660284 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.660265 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd8pj\" (UniqueName: \"kubernetes.io/projected/729c533b-1000-4d81-ad6e-a06a15679e78-kube-api-access-qd8pj\") pod \"migrator-66df5d496b-zvks5\" (UID: \"729c533b-1000-4d81-ad6e-a06a15679e78\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:19.660350 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.660295 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fdkt\" (UniqueName: \"kubernetes.io/projected/85a0105a-b96c-449d-9e91-bd8e457c0a0a-kube-api-access-9fdkt\") pod \"migrator-66df5d496b-ffzcc\" (UID: \"85a0105a-b96c-449d-9e91-bd8e457c0a0a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:19.660793 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.660477 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:19.660793 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.660498 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dd54f845-4vwhq: secret "image-registry-tls" not found Oct 30 18:31:19.660793 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.660553 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls podName:865d20d3-b981-4d85-a56b-4951ae770312 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.660534417 +0000 UTC m=+45.719653403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls") pod "image-registry-6dd54f845-4vwhq" (UID: "865d20d3-b981-4d85-a56b-4951ae770312") : secret "image-registry-tls" not found Oct 30 18:31:19.664282 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.664256 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-srvzm"] Oct 30 18:31:19.682070 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.682035 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:31:19.682213 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.682156 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:31:19.682213 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.682202 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bz2z9\"" Oct 30 18:31:19.713848 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.713820 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-v955r-lpqmq"] Oct 30 18:31:19.723669 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.723638 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:19.735833 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.735810 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.741096 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.741075 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-v955r-lpqmq"] Oct 30 18:31:19.752100 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.752075 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zcslk\"" Oct 30 18:31:19.761517 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761496 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf77bfc-9322-4635-9080-36631d7613df-crio-socket\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.761606 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761542 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:19.761606 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761574 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:19.761712 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761633 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf77bfc-9322-4635-9080-36631d7613df-data-volume\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.761712 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:19.761712 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.761681 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:19.761847 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761723 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd8pj\" (UniqueName: \"kubernetes.io/projected/729c533b-1000-4d81-ad6e-a06a15679e78-kube-api-access-qd8pj\") pod \"migrator-66df5d496b-zvks5\" (UID: \"729c533b-1000-4d81-ad6e-a06a15679e78\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:19.761847 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.761742 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.761723471 +0000 UTC m=+45.820842474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:19.761847 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.761803 2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:19.761998 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.761854 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.761839483 +0000 UTC m=+45.820958482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : configmap references non-existent config key: service-ca.crt Oct 30 18:31:19.761998 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.761875 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.761865701 +0000 UTC m=+45.820984687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:19.761998 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761902 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fdkt\" (UniqueName: \"kubernetes.io/projected/85a0105a-b96c-449d-9e91-bd8e457c0a0a-kube-api-access-9fdkt\") pod \"migrator-66df5d496b-ffzcc\" (UID: \"85a0105a-b96c-449d-9e91-bd8e457c0a0a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:19.761998 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.761944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.762225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.762084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:19.762225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.762116 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmcdg\" (UniqueName: \"kubernetes.io/projected/cbf77bfc-9322-4635-9080-36631d7613df-kube-api-access-fmcdg\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.762225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.762188 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.762225 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.762223 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:19.762418 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.762233 2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:31:19.762418 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.762249 2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-687ccc8fd4-p2sn5: secret "image-registry-tls" not found Oct 30 18:31:19.762418 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.762348 2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:19.762418 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.762385 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls podName:9e6926c1-1692-4111-85b6-319433b9bdfe nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.762375367 +0000 UTC m=+45.821494350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls") pod "image-registry-687ccc8fd4-p2sn5" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe") : secret "image-registry-tls" not found Oct 30 18:31:19.762582 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.762427 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.762414833 +0000 UTC m=+45.821533829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:19.794920 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.794895 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd8pj\" (UniqueName: \"kubernetes.io/projected/729c533b-1000-4d81-ad6e-a06a15679e78-kube-api-access-qd8pj\") pod \"migrator-66df5d496b-zvks5\" (UID: \"729c533b-1000-4d81-ad6e-a06a15679e78\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:19.795315 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.795296 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fdkt\" (UniqueName: \"kubernetes.io/projected/85a0105a-b96c-449d-9e91-bd8e457c0a0a-kube-api-access-9fdkt\") pod \"migrator-66df5d496b-ffzcc\" (UID: \"85a0105a-b96c-449d-9e91-bd8e457c0a0a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:19.833101 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.833078 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:19.840068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.839943 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gwzzt" event={"ID":"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43","Type":"ContainerStarted","Data":"81e3df38de993fa1b226eb25e4fee3ff207c7cf3cd4d8bdd5ffeacc6d51cf4f3"} Oct 30 18:31:19.840068 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.840025 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:19.841687 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.841639 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" event={"ID":"72c54cf4-7294-4d5b-a0f5-61d989cd960f","Type":"ContainerStarted","Data":"94f3da78030e43fe779c402817084fa483c744a7e5bbe4bb96297849036830da"} Oct 30 18:31:19.843463 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.843442 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:31:19.843875 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.843859 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/0.log" Oct 30 18:31:19.843977 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.843888 2584 generic.go:356] "Generic (PLEG): container finished" podID="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" containerID="4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55" exitCode=255 Oct 30 18:31:19.843977 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.843948 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" event={"ID":"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3","Type":"ContainerDied","Data":"4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55"} Oct 30 18:31:19.844114 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.843984 2584 scope.go:122] "RemoveContainer" containerID="155fe592ba0cc001a90293f399f2f33469c7eb8970cdf05bee2ce36800b4394f" Oct 30 18:31:19.844207 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.844191 2584 scope.go:122] "RemoveContainer" containerID="4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55" Oct 30 18:31:19.844392 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.844370 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-c262b_openshift-console-operator(eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" podUID="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" Oct 30 18:31:19.848194 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.848176 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:19.862741 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862714 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:19.862848 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862760 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf77bfc-9322-4635-9080-36631d7613df-data-volume\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.862848 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862792 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.862946 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862861 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.862946 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862897 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.862946 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862927 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.863080 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862962 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmcdg\" (UniqueName: \"kubernetes.io/projected/cbf77bfc-9322-4635-9080-36631d7613df-kube-api-access-fmcdg\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.863080 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.862993 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.863080 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.863012 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.863080 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.863048 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf77bfc-9322-4635-9080-36631d7613df-crio-socket\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.863080 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.863072 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:19.863333 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863191 2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:19.863333 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863240 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.863224312 +0000 UTC m=+45.922343318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:19.863333 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863295 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:19.863333 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863319 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.863310772 +0000 UTC m=+45.922429769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:19.863574 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.863544 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf77bfc-9322-4635-9080-36631d7613df-data-volume\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.863714 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863698 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:31:19.863785 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.863720 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf77bfc-9322-4635-9080-36631d7613df-crio-socket\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.863785 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863749 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:20.363734355 +0000 UTC m=+38.422853354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : configmap "kube-rbac-proxy" not found Oct 30 18:31:19.863785 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863751 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:19.863946 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:19.863793 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:20.36378274 +0000 UTC m=+38.422901736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:19.885698 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.885655 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gwzzt" podStartSLOduration=33.63838169 podStartE2EDuration="36.885641945s" totalImagesPullingTime="3.247260255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:30:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:19.885037913 +0000 UTC m=+37.944156931" watchObservedRunningTime="2026-10-30 18:31:19.885641945 +0000 UTC m=+37.944760948" Oct 30 18:31:19.888748 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.888614 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmcdg\" (UniqueName: \"kubernetes.io/projected/cbf77bfc-9322-4635-9080-36631d7613df-kube-api-access-fmcdg\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:19.924427 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.924253 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:19.930395 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.929288 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" podStartSLOduration=185.663340406 podStartE2EDuration="3m11.929274483s" totalImagesPullingTime="6.265934077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:19.92792857 +0000 UTC m=+37.987047576" watchObservedRunningTime="2026-10-30 18:31:19.929274483 +0000 UTC m=+37.988393488" Oct 30 18:31:19.963995 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.963868 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.963995 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.963934 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.965408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.964361 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.965408 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.964572 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.966855 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.966369 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.966855 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.966560 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.967222 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.967187 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:19.989879 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:19.989853 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:20.063200 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.063163 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:20.152308 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.152089 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5"] Oct 30 18:31:20.157780 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:20.157748 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729c533b_1000_4d81_ad6e_a06a15679e78.slice/crio-c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1 WatchSource:0}: Error finding container c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1: Status 404 returned error can't find the container with id c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1 Oct 30 18:31:20.188789 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.188724 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc"] Oct 30 18:31:20.190796 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:20.190770 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a0105a_b96c_449d_9e91_bd8e457c0a0a.slice/crio-76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626 WatchSource:0}: Error finding container 76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626: Status 404 returned error can't find the container with id 76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626 Oct 30 18:31:20.278435 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.278404 2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-9q4dn"] Oct 30 18:31:20.297585 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.297560 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.302431 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.302407 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-9q4dn"] Oct 30 18:31:20.313595 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.313565 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 18:31:20.313595 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.313566 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 18:31:20.313758 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.313605 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 18:31:20.313758 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.313630 2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 18:31:20.313758 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.313639 2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wkxtp\"" Oct 30 18:31:20.369322 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.369290 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:20.369511 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:20.369374 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:31:20.369511 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.369374 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:20.369511 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:20.369445 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:21.369423865 +0000 UTC m=+39.428542854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : configmap "kube-rbac-proxy" not found Oct 30 18:31:20.369511 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:20.369461 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:20.369511 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:20.369514 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:21.36949911 +0000 UTC m=+39.428618107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:20.379994 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.379970 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-v955r-lpqmq"] Oct 30 18:31:20.382351 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:20.382324 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa532be_e39c_4a3d_bf81_da68387b9703.slice/crio-9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5 WatchSource:0}: Error finding container 9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5: Status 404 returned error can't find the container with id 9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5 Oct 30 18:31:20.470069 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.470037 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s49v\" (UniqueName: \"kubernetes.io/projected/1f3acf05-0627-4917-b471-c27745e87bf0-kube-api-access-2s49v\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.470212 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.470102 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1f3acf05-0627-4917-b471-c27745e87bf0-signing-cabundle\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.470212 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.470189 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1f3acf05-0627-4917-b471-c27745e87bf0-signing-key\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.571619 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.571576 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1f3acf05-0627-4917-b471-c27745e87bf0-signing-key\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.571797 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.571773 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s49v\" (UniqueName: \"kubernetes.io/projected/1f3acf05-0627-4917-b471-c27745e87bf0-kube-api-access-2s49v\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.571906 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.571828 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1f3acf05-0627-4917-b471-c27745e87bf0-signing-cabundle\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.573052 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.573024 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1f3acf05-0627-4917-b471-c27745e87bf0-signing-cabundle\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.575037 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.575010 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1f3acf05-0627-4917-b471-c27745e87bf0-signing-key\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.597748 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.597722 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s49v\" (UniqueName: \"kubernetes.io/projected/1f3acf05-0627-4917-b471-c27745e87bf0-kube-api-access-2s49v\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.632994 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.632964 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:20.664172 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.664123 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:20.854229 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.854126 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:20.860400 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.860357 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:31:20.860797 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.860777 2584 scope.go:122] "RemoveContainer" containerID="4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55" Oct 30 18:31:20.861039 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:20.861011 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-c262b_openshift-console-operator(eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" podUID="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" Oct 30 18:31:20.862785 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.862745 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v955r-lpqmq" event={"ID":"faa532be-e39c-4a3d-bf81-da68387b9703","Type":"ContainerStarted","Data":"0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1"} Oct 30 18:31:20.862879 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.862802 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v955r-lpqmq" event={"ID":"faa532be-e39c-4a3d-bf81-da68387b9703","Type":"ContainerStarted","Data":"9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5"} Oct 30 18:31:20.864112 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.864053 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" event={"ID":"85a0105a-b96c-449d-9e91-bd8e457c0a0a","Type":"ContainerStarted","Data":"76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626"} Oct 30 18:31:20.865281 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.865249 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" event={"ID":"729c533b-1000-4d81-ad6e-a06a15679e78","Type":"ContainerStarted","Data":"c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1"} Oct 30 18:31:20.955555 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.955487 2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-9q4dn"] Oct 30 18:31:20.957932 ip-10-0-130-102 kubenswrapper[2584]: W1030 18:31:20.957880 2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3acf05_0627_4917_b471_c27745e87bf0.slice/crio-37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2 WatchSource:0}: Error finding container 37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2: Status 404 returned error can't find the container with id 37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2 Oct 30 18:31:20.965097 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:20.965057 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-v955r-lpqmq" podStartSLOduration=1.965044241 podStartE2EDuration="1.965044241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:20.964512218 +0000 UTC m=+39.023631248" watchObservedRunningTime="2026-10-30 18:31:20.965044241 +0000 UTC m=+39.024163245" Oct 30 18:31:21.390583 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:21.390547 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:21.390752 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:21.390612 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:21.390752 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:21.390674 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:31:21.390752 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:21.390752 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:23.390731731 +0000 UTC m=+41.449850716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : configmap "kube-rbac-proxy" not found Oct 30 18:31:21.390922 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:21.390792 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:21.390922 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:21.390829 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:23.390818623 +0000 UTC m=+41.449937620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:21.613683 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:21.613594 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:21.873479 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:21.873445 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" event={"ID":"1f3acf05-0627-4917-b471-c27745e87bf0","Type":"ContainerStarted","Data":"909ae8453c38fd841ac32cea8fe95f92cf91e8ff76cb2eabfa3ff3ab8a9afb5c"} Oct 30 18:31:21.874003 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:21.873499 2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" event={"ID":"1f3acf05-0627-4917-b471-c27745e87bf0","Type":"ContainerStarted","Data":"37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2"} Oct 30 18:31:21.924278 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:21.924203 2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" podStartSLOduration=1.9241861999999998 podStartE2EDuration="1.9241862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:21.922704724 +0000 UTC m=+39.981823731" watchObservedRunningTime="2026-10-30 18:31:21.9241862 +0000 UTC m=+39.983305204" Oct 30 18:31:22.243305 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:22.242308 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:22.376887 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:22.376848 2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:22.377060 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:22.376913 2584 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:22.377321 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:22.377306 2584 scope.go:122] "RemoveContainer" containerID="4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55" Oct 30 18:31:22.377534 ip-10-0-130-102 kubenswrapper[2584]: E1030 18:31:22.377514 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-c262b_openshift-console-operator(eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" podUID="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" Oct 30 18:31:22.432605 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:22.432530 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:22.598238 ip-10-0-130-102 kubenswrapper[2584]: I1030 18:31:22.598197 2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:22.829392 ip-10-0-130-102 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 18:31:22.843621 ip-10-0-130-102 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 18:31:22.843850 ip-10-0-130-102 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 18:31:22.843909 ip-10-0-130-102 systemd[1]: kubelet.service: Consumed 3.167s CPU time. Oct 30 18:31:22.854782 ip-10-0-130-102 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:31:22.945319 ip-10-0-130-102 kubenswrapper[5070]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:31:22.945319 ip-10-0-130-102 kubenswrapper[5070]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:31:22.945319 ip-10-0-130-102 kubenswrapper[5070]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:31:22.945319 ip-10-0-130-102 kubenswrapper[5070]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:31:22.947368 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947280 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.947368 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947367 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.947368 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947371 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947375 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947379 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947382 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947384 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947387 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947390 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947392 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947395 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947398 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947401 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947403 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947406 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947409 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947411 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947414 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947417 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947419 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947422 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947424 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.947474 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947428 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947431 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947434 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947436 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947439 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947447 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947450 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947455 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947460 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947463 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947467 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947470 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947473 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947477 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947480 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947483 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947486 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947489 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947492 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.948022 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947495 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947498 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947500 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947503 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947506 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947508 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947511 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947514 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947517 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947520 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947523 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947526 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947534 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947538 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947542 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947545 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947547 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947550 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947553 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947556 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.948505 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947559 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947562 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947565 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947567 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947570 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947574 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947577 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947580 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947583 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947585 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947588 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947591 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947593 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947596 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947599 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947601 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947603 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947606 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947609 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.948999 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947612 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947614 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947617 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947619 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947622 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947625 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947627 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947630 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947632 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947635 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947639 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947643 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947646 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947648 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947651 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947657 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947660 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947663 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947666 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.949515 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947669 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947671 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947674 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947677 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947680 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947682 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.947686 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948460 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948468 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948471 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948474 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948477 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948480 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948483 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948485 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948489 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948492 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948495 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948498 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.949990 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948502 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948507 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948511 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948514 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948518 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948521 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948526 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948529 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948534 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948541 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948545 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948548 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948551 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948553 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948556 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948559 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948561 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948564 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948567 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948570 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.950490 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948572 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948575 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948578 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948580 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948583 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948586 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948590 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948592 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948595 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948598 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948600 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948603 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948605 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948608 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948610 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948613 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948615 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948619 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948621 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.951004 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948624 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948626 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948631 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948634 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948637 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948640 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948643 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948645 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948648 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948651 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948654 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948656 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948659 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948661 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948664 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948666 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948669 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948673 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948677 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.951538 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948679 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948682 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948685 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948688 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948690 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948693 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948695 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948698 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948701 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948703 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948706 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948709 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948712 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948714 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948717 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948721 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948724 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948726 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948729 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948732 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.952064 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948734 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948737 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948740 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948742 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948745 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948747 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948750 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948753 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948755 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948758 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948760 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948763 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948765 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948768 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948772 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.948775 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949314 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949322 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949325 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949329 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.952684 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949332 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949334 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949338 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949341 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949343 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949346 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949349 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949354 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949356 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949359 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949362 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949365 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949367 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949370 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949373 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949376 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949379 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949381 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949384 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.953231 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949386 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949389 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949391 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949394 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949397 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949399 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949402 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949408 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949411 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949414 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949417 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949419 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949425 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949428 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949432 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949435 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949438 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949441 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949444 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.953693 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949446 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949450 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949454 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949459 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949462 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949465 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949468 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949471 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949473 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949476 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949479 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949481 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949484 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949487 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949490 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949493 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949495 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949498 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949501 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949503 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.954169 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949506 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949509 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949512 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949515 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949518 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949520 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949523 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949526 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949528 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949531 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949534 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949536 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949539 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949542 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949545 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949547 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949550 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949552 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949555 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949557 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.954668 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949560 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949562 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949565 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949567 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949570 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949573 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949575 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949578 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949580 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949583 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949585 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949588 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949590 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949592 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949595 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949598 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949600 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949603 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949606 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949608 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.955189 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949611 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949613 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949616 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.949619 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949723 5070 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949735 5070 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949742 5070 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949746 5070 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949751 5070 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949754 5070 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949759 5070 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949763 5070 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949769 5070 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949773 5070 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949776 5070 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949780 5070 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949783 5070 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949786 5070 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949789 5070 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949792 5070 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949795 5070 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949798 5070 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949802 5070 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949805 5070 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:31:22.955699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949808 5070 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949811 5070 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949815 5070 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949819 5070 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949823 5070 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949825 5070 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949829 5070 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949832 5070 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949835 5070 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949838 5070 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949841 5070 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949844 5070 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949849 5070 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949852 5070 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949855 5070 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949858 5070 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949861 5070 flags.go:64] FLAG: --enable-server="true" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949863 5070 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949871 5070 flags.go:64] FLAG: --event-burst="100" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949874 5070 flags.go:64] FLAG: --event-qps="50" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949879 5070 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949882 5070 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949885 5070 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949889 5070 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:31:22.956581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949892 5070 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949895 5070 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949898 5070 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949901 5070 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949904 5070 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949907 5070 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949910 5070 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949913 5070 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949916 5070 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949919 5070 flags.go:64] FLAG: --feature-gates="" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949923 5070 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949926 5070 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949935 5070 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949939 5070 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949942 5070 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949945 5070 flags.go:64] FLAG: --help="false" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949948 5070 flags.go:64] FLAG: --hostname-override="ip-10-0-130-102.ec2.internal" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949951 5070 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949954 5070 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949957 5070 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949961 5070 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949964 5070 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949967 5070 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:31:22.957200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949970 5070 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949973 5070 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949975 5070 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949978 5070 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949981 5070 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949986 5070 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949989 5070 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949992 5070 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949996 5070 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.949999 5070 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950002 5070 flags.go:64] FLAG: --lock-file="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950004 5070 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950008 5070 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950011 5070 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950016 5070 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950019 5070 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950022 5070 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950025 5070 flags.go:64] FLAG: --logging-format="text" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950028 5070 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950033 5070 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950036 5070 flags.go:64] FLAG: --manifest-url="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950039 5070 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950046 5070 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950050 5070 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950054 5070 flags.go:64] FLAG: --max-pods="110" Oct 30 18:31:22.957800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950057 5070 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950060 5070 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950063 5070 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950067 5070 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950070 5070 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950073 5070 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950076 5070 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950083 5070 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950086 5070 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950089 5070 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950092 5070 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950095 5070 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950104 5070 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950107 5070 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950110 5070 flags.go:64] FLAG: --port="10250" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950113 5070 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950116 5070 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076af53a62a845815" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950119 5070 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950122 5070 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950125 5070 flags.go:64] FLAG: --register-node="true" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950128 5070 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950132 5070 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950135 5070 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950138 5070 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950156 5070 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950162 5070 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:31:22.958457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950167 5070 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950170 5070 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950173 5070 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950176 5070 flags.go:64] FLAG: --runonce="false" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950179 5070 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950183 5070 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950186 5070 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950189 5070 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950192 5070 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950195 5070 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950198 5070 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950201 5070 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950204 5070 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950207 5070 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950210 5070 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950213 5070 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950216 5070 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950219 5070 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950224 5070 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950229 5070 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950232 5070 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950235 5070 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950241 5070 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950244 5070 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950248 5070 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:31:22.959330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950251 5070 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950254 5070 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950257 5070 flags.go:64] FLAG: --v="2" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950261 5070 flags.go:64] FLAG: --version="false" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950266 5070 flags.go:64] FLAG: --vmodule="" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950270 5070 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950274 5070 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950404 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950407 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950411 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950414 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950417 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950420 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950423 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950426 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950428 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950431 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950434 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950436 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950439 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950442 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.960134 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950445 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950448 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950450 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950453 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950457 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950459 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950462 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950464 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950467 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950469 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950472 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950475 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950477 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950480 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950483 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950485 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950488 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950490 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950496 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950499 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.960676 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950502 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950504 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950507 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950511 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950514 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950518 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950521 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950524 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950526 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950529 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950532 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950534 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950537 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950539 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950542 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950545 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950548 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950551 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950554 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950556 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.961477 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950560 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950564 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950567 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950569 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950572 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950574 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950577 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950580 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950583 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950585 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950589 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950592 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950594 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950597 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950600 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950603 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950606 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950609 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950612 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.971284 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950614 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950617 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950620 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950622 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950625 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950628 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950630 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950633 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950635 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950639 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950642 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950644 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950647 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950649 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950652 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950654 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950657 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950659 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950662 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.971790 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950665 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950667 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950670 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950673 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950676 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950679 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950681 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950684 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950687 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950689 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950692 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950695 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950698 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950700 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.950725 5070 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950761 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.972280 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950765 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950767 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950770 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950773 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950776 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950778 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950782 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950785 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950788 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950791 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950795 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950798 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950801 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950803 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950806 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950808 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950811 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950814 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950816 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950819 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.972667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950821 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950825 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950828 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950831 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950833 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950836 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950840 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950844 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950847 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950851 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950854 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950856 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950859 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950862 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950864 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950867 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950869 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950872 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950876 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.973229 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950879 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950881 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950884 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950886 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950889 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950891 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950894 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950897 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950899 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950902 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950905 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950908 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950911 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950913 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950917 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950920 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950923 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950926 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950929 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.973711 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950931 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950934 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950936 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950939 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950942 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950945 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950947 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950950 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950953 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950955 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950958 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950960 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950964 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950966 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950969 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950972 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950974 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950977 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950979 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.974222 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950982 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950984 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950987 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950989 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950992 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950995 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.950997 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951000 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951003 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951006 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951009 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951012 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951015 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951018 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951020 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951023 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951026 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951028 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951031 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951034 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.974686 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951036 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951039 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951042 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951044 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951047 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951051 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951054 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.951057 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.955073 5070 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.955088 5070 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.955105 5070 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.955110 5070 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955186 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955192 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955197 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955201 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955204 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955207 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955210 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.975190 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955213 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955216 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955219 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955221 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955224 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955227 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955229 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955232 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955235 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955237 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955240 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955243 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955246 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955249 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955251 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955254 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955257 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955276 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955281 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955285 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.975657 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955288 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955291 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955294 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955297 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955300 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955303 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955305 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955308 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955311 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955314 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955317 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955319 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955322 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955325 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955327 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955330 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955333 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955335 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955338 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955340 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.976180 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955343 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955346 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955348 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955351 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955355 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955359 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955362 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955365 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955368 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955372 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955376 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955379 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955382 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955385 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955388 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955391 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955393 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955396 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955399 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.976749 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955401 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955404 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955407 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955409 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955412 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955414 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955417 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955419 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955422 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955425 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955427 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955430 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955432 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955435 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955438 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955440 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955443 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955445 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955448 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955451 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.977558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955453 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955456 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955459 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955461 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955464 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955467 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955470 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955472 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955475 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955478 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955481 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955484 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955486 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955489 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955492 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955494 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955497 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955499 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955502 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.978417 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955504 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.955537 5070 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955665 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955669 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955673 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955676 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955678 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955681 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955684 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955686 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955689 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955691 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955695 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955698 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955700 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955703 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.978911 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955706 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955709 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955711 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955714 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955717 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955720 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955722 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955725 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955728 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955730 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955732 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955735 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955738 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955740 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955743 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955745 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955748 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955751 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955753 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955756 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.979372 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955758 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955761 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955764 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955769 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955772 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955775 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955777 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955780 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955783 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955787 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955789 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955792 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955794 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955797 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955800 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955802 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955805 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955807 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955811 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.980000 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955815 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955818 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955821 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955823 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955828 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955831 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955835 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955839 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955843 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955847 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955851 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955855 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955859 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955863 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955867 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955871 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955876 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955880 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955884 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.980558 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955888 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955892 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955896 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955900 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955905 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955910 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955914 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955918 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955923 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955928 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955932 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955936 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955940 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955944 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955948 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955951 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955953 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955956 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955960 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955964 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.981302 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955968 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955972 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955976 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955980 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955985 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955989 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955994 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.955998 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956002 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956006 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956010 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956015 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956020 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956024 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.956066 5070 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:31:22.982187 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956119 5070 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956126 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956132 5070 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956135 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956140 5070 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956159 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956163 5070 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956167 5070 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956171 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956176 5070 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956180 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956185 5070 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956189 5070 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956194 5070 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956198 5070 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956202 5070 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956207 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956212 5070 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956216 5070 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:31:22.982843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956221 5070 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956223 5070 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956226 5070 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956229 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956231 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956234 5070 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956238 5070 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956240 5070 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956243 5070 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956246 5070 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956249 5070 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956251 5070 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956254 5070 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956259 5070 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956262 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956264 5070 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956268 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956270 5070 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956273 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:31:22.983689 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956276 5070 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956278 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956281 5070 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956283 5070 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956286 5070 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956289 5070 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956293 5070 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956297 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956300 5070 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956302 5070 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956305 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956308 5070 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956311 5070 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956313 5070 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956316 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956319 5070 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956321 5070 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956324 5070 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956327 5070 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956329 5070 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:31:22.984594 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956332 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956335 5070 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956337 5070 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956340 5070 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956343 5070 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956348 5070 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956354 5070 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956357 5070 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956360 5070 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956363 5070 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956367 5070 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956370 5070 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956372 5070 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956375 5070 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956377 5070 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956380 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956383 5070 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956385 5070 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956392 5070 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956395 5070 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:31:22.985531 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956398 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:31:22.985450 ip-10-0-130-102 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956402 5070 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956405 5070 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956408 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956410 5070 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956413 5070 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956416 5070 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956419 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956421 5070 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956424 5070 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956427 5070 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956429 5070 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956432 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956435 5070 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956437 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956440 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956443 5070 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956445 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956449 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:31:22.986521 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956452 5070 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956455 5070 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956457 5070 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956460 5070 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956463 5070 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956466 5070 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956468 5070 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956472 5070 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:22.956475 5070 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.956526 5070 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.957456 5070 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.957511 5070 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.957677 5070 server.go:1014] "Starting client certificate rotation" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.957812 5070 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.957861 5070 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:25:42 +0000 UTC" deadline="2028-07-06 11:27:39.399604412 +0000 UTC" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.957888 5070 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14752h56m16.441718767s" Oct 30 18:31:22.987459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.958088 5070 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:31:22.988194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.960296 5070 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:31:22.988194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.961671 5070 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:31:22.988194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.963317 5070 log.go:25] "Validated CRI v1 image API" Oct 30 18:31:22.988194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.963987 5070 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:31:22.988194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.966081 5070 fs.go:135] Filesystem UUIDs: map[022651ab-6fe0-445f-9e4b-cf87ff853eb2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a8176a86-0e55-4cd7-b5a8-b03c227dde75:/dev/nvme0n1p4] Oct 30 18:31:22.988464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.966099 5070 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1/userdata/shm major:0 minor:289 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2/userdata/shm major:0 minor:362 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/40655e2cdd4d2c280ec2b728030e86b7e2cbc2f7b9e6a7a4eb0ae15d0d1b9cef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/40655e2cdd4d2c280ec2b728030e86b7e2cbc2f7b9e6a7a4eb0ae15d0d1b9cef/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626/userdata/shm major:0 minor:330 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1/userdata/shm major:0 minor:316 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c11638a51ea397d7f9f3bddbd6ad0601fc716e232571f4e369d7c0b8d107fce8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c11638a51ea397d7f9f3bddbd6ad0601fc716e232571f4e369d7c0b8d107fce8/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e9ec10695ea7fed16e26429e0a0845446487a7e50122d5919342abac635526a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e9ec10695ea7fed16e26429e0a0845446487a7e50122d5919342abac635526a1/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06297482-f53e-497f-8012-1c6a55b1462b/volumes/kubernetes.io~projected/kube-api-access-qdzm6:{mountpoint:/var/lib/kubelet/pods/06297482-f53e-497f-8012-1c6a55b1462b/volumes/kubernetes.io~projected/kube-api-access-qdzm6 major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0fae4e48-2851-4c67-9615-ad86f9795397/volumes/kubernetes.io~projected/kube-api-access-68rjt:{mountpoint:/var/lib/kubelet/pods/0fae4e48-2851-4c67-9615-ad86f9795397/volumes/kubernetes.io~projected/kube-api-access-68rjt major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10ed4e5d-002c-48ca-ad1e-199694eef36b/volumes/kubernetes.io~projected/kube-api-access-5zshw:{mountpoint:/var/lib/kubelet/pods/10ed4e5d-002c-48ca-ad1e-199694eef36b/volumes/kubernetes.io~projected/kube-api-access-5zshw major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11d10c3e-223f-41ef-ac3f-cd428bf43ebd/volumes/kubernetes.io~projected/kube-api-access-fvpks:{mountpoint:/var/lib/kubelet/pods/11d10c3e-223f-41ef-ac3f-cd428bf43ebd/volumes/kubernetes.io~projected/kube-api-access-fvpks major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13093b2f-a782-4521-b80f-f4ee194af653/volumes/kubernetes.io~projected/kube-api-access-5l8qg:{mountpoint:/var/lib/kubelet/pods/13093b2f-a782-4521-b80f-f4ee194af653/volumes/kubernetes.io~projected/kube-api-access-5l8qg major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13093b2f-a782-4521-b80f-f4ee194af653/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/13093b2f-a782-4521-b80f-f4ee194af653/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9/volumes/kubernetes.io~projected/kube-api-access-dcbsv:{mountpoint:/var/lib/kubelet/pods/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9/volumes/kubernetes.io~projected/kube-api-access-dcbsv major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9/volumes/kubernetes.io~secret/serving-cert major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~projected/kube-api-access-zpkgn:{mountpoint:/var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~projected/kube-api-access-zpkgn major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~secret/default-certificate major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~secret/stats-auth major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f3acf05-0627-4917-b471-c27745e87bf0/volumes/kubernetes.io~projected/kube-api-access-2s49v:{mountpoint:/var/lib/kubelet/pods/1f3acf05-0627-4917-b471-c27745e87bf0/volumes/kubernetes.io~projected/kube-api-access-2s49v major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f3acf05-0627-4917-b471-c27745e87bf0/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/1f3acf05-0627-4917-b471-c27745e87bf0/volumes/kubernetes.io~secret/signing-key major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30e186ca-854c-417a-96ed-3a660db8b8f4/volumes/kubernetes.io~projected/kube-api-access-xwfcw:{mountpoint:/var/lib/kubelet/pods/30e186ca-854c-417a-96ed-3a660db8b8f4/volumes/kubernetes.io~projected/kube-api-access-xwfcw major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30e186ca-854c-417a-96ed-3a660db8b8f4/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/30e186ca-854c-417a-96ed-3a660db8b8f4/volumes/kubernetes.io~secret/serving-cert major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/399ea207-fdb0-4a0a-a841-b113cffc4c3e/volumes/kubernetes.io~projected/kube-api-access-fd8rj:{mountpoint:/var/lib/kubelet/pods/399ea207-fdb0-4a0a-a841-b113cffc4c3e/volumes/kubernetes.io~projected/kube-api-access-fd8rj major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/463c6228-727e-4d4f-9b80-1db615657b4f/volumes/kubernetes.io~projected/kube-api-access-fd977:{mountpoint:/var/lib/kubelet/pods/463c6228-727e-4d4f-9b80-1db615657b4f/volumes/kubernetes.io~projected/kube-api-access-fd977 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volumes/kubernetes.io~projected/kube-api-access-8zhgq:{mountpoint:/var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volumes/kubernetes.io~projected/kube-api-access-8zhgq major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bb762ff-7409-4218-8421-d0b376a46f98/volumes/kubernetes.io~projected/kube-api-access-87hnt:{mountpoint:/var/lib/kubelet/pods/4bb762ff-7409-4218-8421-d0b376a46f98/volumes/kubernetes.io~projected/kube-api-access-87hnt major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43/volumes/kubernetes.io~projected/kube-api-access-j6qqs:{mountpoint:/var/lib/kubelet/pods/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43/volumes/kubernetes.io~projected/kube-api-access-j6qqs major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/56a08134-6094-4349-b252-ccf6d0aa8774/volumes/kubernetes.io~projected/kube-api-access-bzw9c:{mountpoint:/var/lib/kubelet/pods/56a08134-6094-4349-b252-ccf6d0aa8774/volumes/kubernetes.io~projected/kube-api-access-bzw9c major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68a3d167-dbc7-4155-aee3-e056b558ab8a/volumes/kubernetes.io~projected/kube-api-access-frs5z:{mountpoint:/var/lib/kubelet/pods/68a3d167-dbc7-4155-aee3-e056b558ab8a/volumes/kubernetes.io~projected/kube-api-access-frs5z major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68a3d167-dbc7-4155-aee3-e056b558ab8a/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/68a3d167-dbc7-4155-aee3-e056b558ab8a/volumes/kubernetes.io~secret/cert major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/729c533b-1000-4d81-ad6e-a06a15679e78/volumes/kubernetes.io~projected/kube-api-access-qd8pj:{mountpoint:/var/lib/kubelet/pods/729c533b-1000-4d81-ad6e-a06a15679e78/volumes/kubernetes.io~projected/kube-api-access-qd8pj major:0 minor:308 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72c54cf4-7294-4d5b-a0f5-61d989cd960f/volumes/kubernetes.io~projected/kube-api-access-652nf:{mountpoint:/var/lib/kubelet/pods/72c54cf4-7294-4d5b-a0f5-61d989cd960f/volumes/kubernetes.io~projected/kube-api-access-652nf major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/739cc6ce-77da-492c-9cfc-6bfe5c50dd56/volumes/kubernetes.io~projected/kube-api-access-f7rmv:{mountpoint:/var/lib/kubelet/pods/739cc6ce-77da-492c-9cfc-6bfe5c50dd56/volumes/kubernetes.io~projected/kube-api-access-f7rmv major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79d5b339-6665-4fbe-92db-77fd202119d2/volumes/kubernetes.io~projected/kube-api-access-tw4m7:{mountpoint:/var/lib/kubelet/pods/79d5b339-6665-4fbe-92db-77fd202119d2/volumes/kubernetes.io~projected/kube-api-access-tw4m7 major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85a0105a-b96c-449d-9e91-bd8e457c0a0a/volumes/kubernetes.io~projected/kube-api-access-9fdkt:{mountpoint:/var/lib/kubelet/pods/85a0105a-b96c-449d-9e91-bd8e457c0a0a/volumes/kubernetes.io~projected/kube-api-access-9fdkt major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~projected/kube-api-access-hnlkc:{mountpoint:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~projected/kube-api-access-hnlkc major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b4ec318-a75a-4dd0-9da0-ffc41310b641/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/8b4ec318-a75a-4dd0-9da0-ffc41310b641/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9c695b41-2900-4b0d-832b-bb35456eb816/volumes/kubernetes.io~projected/kube-api-access-mcsbm:{mountpoint:/var/lib/kubelet/pods/9c695b41-2900-4b0d-832b-bb35456eb816/volumes/kubernetes.io~projected/kube-api-access-mcsbm major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~projected/kube-api-access-8m8gm:{mountpoint:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~projected/kube-api-access-8m8gm major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9f0e7740-17a0-460e-9407-5650fa708046/volumes/kubernetes.io~projected/kube-api-access-h5scv:{mountpoint:/var/lib/kubelet/pods/9f0e7740-17a0-460e-9407-5650fa708046/volumes/kubernetes.io~projected/kube-api-access-h5scv major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a04618ce-f5a4-49ae-8fee-ffb082167b8c/volumes/kubernetes.io~projected/kube-api-access-xhgb4:{mountpoint:/var/lib/kubelet/pods/a04618ce-f5a4-49ae-8fee-ffb082167b8c/volumes/kubernetes.io~projected/kube-api-access-xhgb4 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~projected/kube-api-access-hgmhk:{mountpoint:/var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~projected/kube-api-access-hgmhk major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbf77bfc-9322-4635-9080-36631d7613df/volumes/kubernetes.io~projected/kube-api-access-fmcdg:{mountpoint:/var/lib/kubelet/pods/cbf77bfc-9322-4635-9080-36631d7613df/volumes/kubernetes.io~projected/kube-api-access-fmcdg major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf9ba972-af6a-4cda-a59b-f51874d49aae/volumes/kubernetes.io~projected/kube-api-access-dw8z6:{mountpoint:/var/lib/kubelet/pods/cf9ba972-af6a-4cda-a59b-f51874d49aae/volumes/kubernetes.io~projected/kube-api-access-dw8z6 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/deaa4bf3-e4f9-4b22-9bda-eba0080c0697/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/deaa4bf3-e4f9-4b22-9bda-eba0080c0697/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/volumes/kubernetes.io~projected/kube-api-access-zt78g:{mountpoint:/var/lib/kubelet/pods/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/volumes/kubernetes.io~projected/kube-api-access-zt78g major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/volumes/kubernetes.io~secret/serving-cert major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/faa532be-e39c-4a3d-bf81-da68387b9703/volumes/kubernetes.io~projected/kube-api-access-755x7:{mountpoint:/var/lib/kubelet/pods/faa532be-e39c-4a3d-bf81-da68387b9703/volumes/kubernetes.io~projected/kube-api-access-755x7 major:0 minor:341 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/8a0fbd6cb0f31cb94b7690366a086ed230f54bc04b706536ffad4ea81bd24ac2/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/4c526b7dae9ee72bfe2a9f263b1514229a78d4e1f09db9f97dd5478567f6cca2/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/e755dd42c3d7fb77e74c060daf73976317a7ca30a94f2207711ec695e7641295/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/b6122dc834c29c966d0f6cc7bdbb5e97e1dbc7193e253ec3f7711edda36e533f/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/6fa16f31d16e929e846d898f75d5e7a43f651bfb941833136e69d0a335a39d1f/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/20d93a018fd0a11dfd19a8337307be426be797125b5714f03efca5c1fc3d061e/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/2a8b728068024d19f2362fda4b6b6faff221a4f3aef0064d7c17aee91ad9493d/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/16d865994bacf1038fb321e87f740580de089917e585bd4dc0b6cc0b46fa8cd6/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/a0e708ea17877180376dbf4b7a012c246ed488bd8a66e3d681fd19798f34913b/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/0eb2e941e49c83f54cb2d699bba72dfe153752cfc3ad2ef3cfed56bb7b9950f3/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/203d5544ce862ec263df659c736f855f4b590e3e6cec8626fe341859aea86d83/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/59d1c1b4202e2b677bb473279da710584ca009b6f7eaccba7ab444bbd074d859/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/d3bc2c882fcd8b25433922a622e507720f3b8cd098366ae60b4159ff8de27682/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/0a385f71d832484b01c3dcdbfe1ad89f0a6e0c1aa77650bc54472984defff01d/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/2e464c5e485cc1660926f655ba54f0a91ac4b021431e052edf746a0f4ae32543/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-163:{mountpoint:/var/lib/containers/storage/overlay/cd4820de7600843980e9c9c01df1678e279760f7ab956e1282cbb15fd3e93e57/merged major:0 minor:163 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/77e21447726142a90fb772255d831870e2677f98f116df5a7c9594b2b4018d06/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/f672245ca6ff614f9a6d601943490b3adb8fe88189cdf0d097b603b404381fac/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/744f12126218ff064ed8075d652aa4575aa63e67a90fb0e7d0c67f7dc64804d8/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/e789f44c3df46070ea9ffb878178ac1989fe8f859a9bdb6908ac586eae05f009/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/337493ba1aee442b1cd650bcdd302137094689d92b869ff38f5a0e94beaacf72/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/12591895f61e24d72a8046837e7dc11ae8fd4a6a5ce30bf3c9330e2a02664ed3/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/f82765afd8add7541f5d0432bcf92abf7864705449790a03d5b55f57ec0d056e/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/0931a131a14370c92f559ba80dc7c152670274b2da0201923d7454122fb54617/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/acecbc713a2c04bfdcd14d9f3f69a733d234dcc9707015f62e63201d4f51beb4/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/e3fb0cf4504799fb0fb7059934978fc41959c6688f699f02009ee13dfba3513e/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/9f71b0122d413e644e27d0dc3bef1446c8371b884553670cdaece0e6c0b8712e/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/10a88b1fb8f9386285f3fae90813d44c8c39063f0ca91831ffa928825b7bb3ad/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/1768a006ffa0432b9e05cd7f2dbc75e3e9ff9cc9689d2b6b1644844eabbeb302/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/dc2227f47ca71d66d3b74bf91103955500f7422ae6adf89d17fb3589f983e1ae/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/bc410bfeb62cbd7a1d701892845670b2a22d56ea22eb6036e2951f28b7cf627b/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/5ea1c36dd703c31f2b68c3f1f5bf4fdb139a6083919345d3f63630347a60c02a/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/154565fca7cb0cd09c801ba197b0150975b7325666d1c753af13d9f4628d765e/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/c8b54012f0a176505f125a1530bbb5b02ff2d51e3e82ba1a4a9a29648d839da3/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/6592ca86069796ac622bcc7543dd8c72b6f766036ac3bbb5cc39a892accf3161/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/4b30699cfd635ac1e2187422904093e6fb3c55e3d8249511701e2b6b12eee143/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/da5405b3e7bcaf5a06feb0340d12d30f3901b3022551d17faaf03bbeac111dea/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/76021210a37f1fdfd9fdceb9d5efb9be1068c95ec9ac21b3704e7250d014a54d/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/d849fcd6b9fe1adcef2e652d1cad90faacfdf7fc8870224323df9739ffbdaeb0/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/420a603e86de4a39cd99ccf041c595c8691e5fb8eb517b1e4d7d320d44396898/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/8031088557a304e5cba7e04ec11604b1e80387526d7e6d70145569b7281f2056/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/16eb2d6b85f4ac6b57b65e3844042813f9c4030004015dd249d9016a72263dbb/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/7309d0f19a664b426cff91d9aff86b3a3431ccc9646c012110de722125955b23/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/3282f1b154276d63b5a81010ca5ecf73dcfeb4e685fd7631bb1f3551dc625d52/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/adda128aa0c51e729d796f22c31fb3a387867e71b90c9980ad072527796007f8/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/778b14b475fde50597582d2328a9ccce71e4eba6a41a30437f4bb8a7fcdf5748/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/92d0e2a7fc1fb6ef3ad1cee62356e32131e5979f9b3fd4ed68939f986904c4c1/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/2333609bd1fad6f3407efaf73bed4e75c3fa05456e98d0e86e834fbe5af15749/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/1432a3b0305f3433b62a3a378595f34d8549b3fe76555fd6c66c4b512488c220/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/48f729d32d5a7e94e04621aa38c0731d60c8427cd0258faaca626f421b066556/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/c26cb6ccf820da541f5d1c96d2bd05f449eab894ca9d333582d244eaa4b993b9/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/a8f5dde17899d367b8553cd283b1af71d4eb6626dcac4ef64d3d3b7f1e457f54/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/cc0a578d9747911ad9236f6748edd068d4eb8f7408c3bed22e05ce1a90e37bae/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/1613993f429fd363d02b6bf7de970720ee2913cc7908bca9218976c7e9cc4e8e/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/c5dcec2598bde00599fcba945dd9ebd147384c84a4368ce3051e64bfa5882eb0/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/6b7ca7af5148af206eec1be23b3da44257bd2b00f33122d9e8125bacdaf7663a/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/927e2cbf104766e3fdd4108dcc952c20316ce9e5fea97c0c2600605a02b09fcf/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/7937de6e45227a9517b4ed9e9f42d6d26b0d988e98895081d26f3f22d71cff98/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/b5eff27912b955ca71dbcc81ae2dffca2c92ef1c8be23bca32f461608616f184/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-89:{mountpoint:/var/lib/containers/storage/overlay/5ab8ac3f1f03ddf361d1d12ead3d2555d4b41812748c058ffe7bb09b17d9221c/merged major:0 minor:89 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/f09f1bd0434f36163b25cd4139c50fa1e0cc81e4f89ec30ff74a52e3fc6469d5/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/238df96bd623c60a493ce2b5c7e0b3a44815f24a61ab1f4529868582036c6aae/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.974450 5070 manager.go:217] Machine: {Timestamp:2026-10-30 18:31:22.973957807 +0000 UTC m=+0.089091714 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3116460 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cc837afda1383576da86a9fc14cc8 SystemUUID:ec2cc837-afda-1383-576d-a86a9fc14cc8 BootID:cc18319c-eabf-43a4-89c2-8671d813ec96 Filesystems:[{Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/739cc6ce-77da-492c-9cfc-6bfe5c50dd56/volumes/kubernetes.io~projected/kube-api-access-f7rmv DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/30e186ca-854c-417a-96ed-3a660db8b8f4/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4c45c9279df6c240fe111797c0808fdeaaf162a7c7ee0fac2896264caa137563/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85a0105a-b96c-449d-9e91-bd8e457c0a0a/volumes/kubernetes.io~projected/kube-api-access-9fdkt DeviceMajor:0 DeviceMinor:309 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/399ea207-fdb0-4a0a-a841-b113cffc4c3e/volumes/kubernetes.io~projected/kube-api-access-fd8rj DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f9e74b5d698326d5a8d4eae9bd4b1f5469923490ba494b05b6b5485f6e12c880/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9c695b41-2900-4b0d-832b-bb35456eb816/volumes/kubernetes.io~projected/kube-api-access-mcsbm DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/729c533b-1000-4d81-ad6e-a06a15679e78/volumes/kubernetes.io~projected/kube-api-access-qd8pj DeviceMajor:0 DeviceMinor:308 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9f0e7740-17a0-460e-9407-5650fa708046/volumes/kubernetes.io~projected/kube-api-access-h5scv DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/faa532be-e39c-4a3d-bf81-da68387b9703/volumes/kubernetes.io~projected/kube-api-access-755x7 DeviceMajor:0 DeviceMinor:341 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/272f2abecd95f37eea5fd9b91c4e4caea3f82b3742218151b649b7f43951afcf/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ecdb5c7b9b6e632b662fd47b6e74bcd3ed585d9ebd793819c024a640748a6843/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a04618ce-f5a4-49ae-8fee-ffb082167b8c/volumes/kubernetes.io~projected/kube-api-access-xhgb4 DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/76f62ccb4cf90f9bc0adbb374bff1cb46740be1004162ebdb29249649d2dc626/userdata/shm DeviceMajor:0 DeviceMinor:330 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/56a08134-6094-4349-b252-ccf6d0aa8774/volumes/kubernetes.io~projected/kube-api-access-bzw9c DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf9ba972-af6a-4cda-a59b-f51874d49aae/volumes/kubernetes.io~projected/kube-api-access-dw8z6 DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volumes/kubernetes.io~projected/kube-api-access-8zhgq DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c024f6cdde7c63238a73d0e2aedddb6517d352f13d1cfaff95ad474f2475a2d1/userdata/shm DeviceMajor:0 DeviceMinor:316 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1f3acf05-0627-4917-b471-c27745e87bf0/volumes/kubernetes.io~projected/kube-api-access-2s49v DeviceMajor:0 DeviceMinor:361 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-89 DeviceMajor:0 DeviceMinor:89 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc8e6bdb7e940b23fe90701b34529d0166eab5b1ac6567c60159a85b1c65cf66/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/44763eef55d576fcf8b3534063073e2ce795c3451fefad0dcae6f5e5887538f1/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/edd935dc3c6d51fa062028d3067122a5357993a8840522c8f15ae9976db1f874/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a482ceb3b7e8effba21b1f9f184ab44140add28266b42be89168bcb9fa243ee9/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/46a5ae95b24f39a5b5f2d98c7988ca2037d22c52d5c6c62bbfc00ad06f1533fc/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b4ec318-a75a-4dd0-9da0-ffc41310b641/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/79d5b339-6665-4fbe-92db-77fd202119d2/volumes/kubernetes.io~projected/kube-api-access-tw4m7 DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/06297482-f53e-497f-8012-1c6a55b1462b/volumes/kubernetes.io~projected/kube-api-access-qdzm6 DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6762df70e8e3ed096d19ba77ccfcf544b00086bd9c201b354603468be5a68211/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c11638a51ea397d7f9f3bddbd6ad0601fc716e232571f4e369d7c0b8d107fce8/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/77b2322703f14a23ad4567f3378cb97f115df7b409d6bfdd4dcba4f9f98abc04/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/68a3d167-dbc7-4155-aee3-e056b558ab8a/volumes/kubernetes.io~projected/kube-api-access-frs5z DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f3acf05-0627-4917-b471-c27745e87bf0/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:356 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/55c2240eeb237c60d786a13d7e6fb3eabbcf3a8acefeaff36cbd2c5ebb7d3be0/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/72c54cf4-7294-4d5b-a0f5-61d989cd960f/volumes/kubernetes.io~projected/kube-api-access-652nf DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b6951a2ca7bd6caaf70ca2375ca87202329e81ec2a2e67356beb74ebbe610c1/userdata/shm DeviceMajor:0 DeviceMinor:289 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/47df06d1-8c08-47c2-9e1f-10636f6bf035/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e9ec10695ea7fed16e26429e0a0845446487a7e50122d5919342abac635526a1/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11d10c3e-223f-41ef-ac3f-cd428bf43ebd/volumes/kubernetes.io~projected/kube-api-access-fvpks DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/10ed4e5d-002c-48ca-ad1e-199694eef36b/volumes/kubernetes.io~projected/kube-api-access-5zshw DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b60126a0-255e-4b6d-9166-18c92777ddc5/volumes/kubernetes.io~projected/kube-api-access-hgmhk DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~projected/kube-api-access-hnlkc DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/volumes/kubernetes.io~projected/kube-api-access-zpkgn DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/40655e2cdd4d2c280ec2b728030e86b7e2cbc2f7b9e6a7a4eb0ae15d0d1b9cef/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d499755b3d3005a5e2bff874848db92001b6bc7c67549dff0d2bb75dfd52834c/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/68a3d167-dbc7-4155-aee3-e056b558ab8a/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-163 DeviceMajor:0 DeviceMinor:163 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes/kubernetes.io~projected/kube-api-access-8m8gm DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43/volumes/kubernetes.io~projected/kube-api-access-j6qqs DeviceMajor:0 DeviceMinor:290 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/741b422a384c14626f773a90a1725b282b11cf1a9a0846abf3c375a6475ded3e/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/95c32e5ca2ea2ebc72d3fb6d79689add87064b7cd205c4d7a1763ba6d1cc9b4d/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0fae4e48-2851-4c67-9615-ad86f9795397/volumes/kubernetes.io~projected/kube-api-access-68rjt DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4bb762ff-7409-4218-8421-d0b376a46f98/volumes/kubernetes.io~projected/kube-api-access-87hnt DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2420ef5e361974c3308c82ff109e1d5de54bca228e197e7916b6a7dbdb0400cf/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/30e186ca-854c-417a-96ed-3a660db8b8f4/volumes/kubernetes.io~projected/kube-api-access-xwfcw DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/13093b2f-a782-4521-b80f-f4ee194af653/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/13093b2f-a782-4521-b80f-f4ee194af653/volumes/kubernetes.io~projected/kube-api-access-5l8qg DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/volumes/kubernetes.io~projected/kube-api-access-zt78g DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cbf77bfc-9322-4635-9080-36631d7613df/volumes/kubernetes.io~projected/kube-api-access-fmcdg DeviceMajor:0 DeviceMinor:340 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37a24ad6d4e05c10c337af6511f9aa750145347401d2836aa119ac2b334294a2/userdata/shm DeviceMajor:0 DeviceMinor:362 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9/volumes/kubernetes.io~projected/kube-api-access-dcbsv DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/463c6228-727e-4d4f-9b80-1db615657b4f/volumes/kubernetes.io~projected/kube-api-access-fd977 DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/deaa4bf3-e4f9-4b22-9bda-eba0080c0697/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:286 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:272f2abecd95f37 MacAddress:a2:44:ad:7b:70:98 Speed:10000 Mtu:8901} {Name:2b6951a2ca7bd6c MacAddress:52:b4:5f:8b:5b:f8 Speed:10000 Mtu:8901} {Name:37a24ad6d4e05c1 MacAddress:32:de:8e:4e:b7:41 Speed:10000 Mtu:8901} {Name:40655e2cdd4d2c2 MacAddress:8e:96:65:36:de:c9 Speed:10000 Mtu:8901} {Name:44763eef55d576f MacAddress:ba:aa:e5:99:b2:69 Speed:10000 Mtu:8901} {Name:46a5ae95b24f39a MacAddress:4a:42:a2:7d:94:3f Speed:10000 Mtu:8901} {Name:4c45c9279df6c24 MacAddress:4e:a5:3d:76:4e:c6 Speed:10000 Mtu:8901} {Name:55c2240eeb237c6 MacAddress:2e:4a:74:ef:14:5b Speed:10000 Mtu:8901} {Name:6762df70e8e3ed0 MacAddress:da:15:c9:31:2c:34 Speed:10000 Mtu:8901} {Name:76f62ccb4cf90f9 MacAddress:62:57:14:80:2e:39 Speed:10000 Mtu:8901} {Name:9588ef34e32c3ec MacAddress:b6:24:ab:5e:2c:6f Speed:10000 Mtu:8901} {Name:a482ceb3b7e8eff MacAddress:d2:b0:a7:e2:a4:55 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f5:40:cf:e7 Speed:0 Mtu:9001} {Name:br-int MacAddress:82:6f:ec:b7:4e:23 Speed:0 Mtu:8901} {Name:c024f6cdde7c632 MacAddress:02:cf:9e:0e:f6:b0 Speed:10000 Mtu:8901} {Name:c11638a51ea397d MacAddress:8a:4d:c6:fe:a2:4c Speed:10000 Mtu:8901} {Name:d499755b3d3005a MacAddress:c6:fd:d5:fd:da:c6 Speed:10000 Mtu:8901} {Name:e9ec10695ea7fed MacAddress:62:a5:3e:da:86:72 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f5:40:cf:e7 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:8e:17:be:2c:80:fd Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:f6:a8:eb:36:20:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.974800 5070 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.974841 5070 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975128 5070 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975170 5070 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975291 5070 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975303 5070 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975320 5070 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975347 5070 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975374 5070 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975570 5070 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975675 5070 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975707 5070 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975716 5070 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975726 5070 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975735 5070 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.975744 5070 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.977432 5070 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.977451 5070 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.978302 5070 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.980913 5070 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.980925 5070 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981016 5070 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981208 5070 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981234 5070 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981323 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981335 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981341 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981347 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981353 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981359 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981365 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981370 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981377 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981383 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981391 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981401 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981424 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.981429 5070 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.984876 5070 server.go:1263] "Started kubelet" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.984944 5070 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.985065 5070 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.985186 5070 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.986121 5070 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.987000 5070 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:31:22.988881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.988596 5070 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.989592 5070 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.989675 5070 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:25:43 +0000 UTC" deadline="2028-08-12 00:00:39.992192604 +0000 UTC" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.989718 5070 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15629h29m17.002478281s" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:22.989752 5070 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-102.ec2.internal\" not found" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.989803 5070 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.989811 5070 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:31:22.991484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.990068 5070 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:31:22.995910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995705 5070 factory.go:55] Registering systemd factory Oct 30 18:31:22.995910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995731 5070 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:31:22.995910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995853 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47df06d1-8c08-47c2-9e1f-10636f6bf035" volumeName="kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-env-overrides" seLinuxMountContext="" Oct 30 18:31:22.995910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995894 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995930 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faa532be-e39c-4a3d-bf81-da68387b9703" volumeName="kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995974 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11d10c3e-223f-41ef-ac3f-cd428bf43ebd" volumeName="kubernetes.io/projected/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-kube-api-access-fvpks" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.995992 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3acf05-0627-4917-b471-c27745e87bf0" volumeName="kubernetes.io/configmap/1f3acf05-0627-4917-b471-c27745e87bf0-signing-cabundle" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996021 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="463c6228-727e-4d4f-9b80-1db615657b4f" volumeName="kubernetes.io/empty-dir/463c6228-727e-4d4f-9b80-1db615657b4f-tmp-dir" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996032 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47df06d1-8c08-47c2-9e1f-10636f6bf035" volumeName="kubernetes.io/projected/47df06d1-8c08-47c2-9e1f-10636f6bf035-kube-api-access-8zhgq" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996043 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bb762ff-7409-4218-8421-d0b376a46f98" volumeName="kubernetes.io/projected/4bb762ff-7409-4218-8421-d0b376a46f98-kube-api-access-87hnt" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996055 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" volumeName="kubernetes.io/projected/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-kube-api-access-f7rmv" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996067 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="deaa4bf3-e4f9-4b22-9bda-eba0080c0697" volumeName="kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996076 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996084 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf77bfc-9322-4635-9080-36631d7613df" volumeName="kubernetes.io/empty-dir/cbf77bfc-9322-4635-9080-36631d7613df-data-volume" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996095 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30e186ca-854c-417a-96ed-3a660db8b8f4" volumeName="kubernetes.io/configmap/30e186ca-854c-417a-96ed-3a660db8b8f4-config" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996104 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30e186ca-854c-417a-96ed-3a660db8b8f4" volumeName="kubernetes.io/projected/30e186ca-854c-417a-96ed-3a660db8b8f4-kube-api-access-xwfcw" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996117 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" volumeName="kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996132 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996164 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9c695b41-2900-4b0d-832b-bb35456eb816" volumeName="kubernetes.io/projected/9c695b41-2900-4b0d-832b-bb35456eb816-kube-api-access-mcsbm" seLinuxMountContext="" Oct 30 18:31:22.996168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996177 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996189 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996205 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" volumeName="kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-config" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996256 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19c4a917-c2ba-46e3-8b7f-d528b1c26bc9" volumeName="kubernetes.io/projected/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-kube-api-access-dcbsv" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996271 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3acf05-0627-4917-b471-c27745e87bf0" volumeName="kubernetes.io/projected/1f3acf05-0627-4917-b471-c27745e87bf0-kube-api-access-2s49v" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996289 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996300 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0fae4e48-2851-4c67-9615-ad86f9795397" volumeName="kubernetes.io/projected/0fae4e48-2851-4c67-9615-ad86f9795397-kube-api-access-68rjt" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996317 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" volumeName="kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996328 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06297482-f53e-497f-8012-1c6a55b1462b" volumeName="kubernetes.io/configmap/06297482-f53e-497f-8012-1c6a55b1462b-telemetry-config" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996342 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10ed4e5d-002c-48ca-ad1e-199694eef36b" volumeName="kubernetes.io/projected/10ed4e5d-002c-48ca-ad1e-199694eef36b-kube-api-access-5zshw" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996350 5070 factory.go:153] Registering CRI-O factory Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996359 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13093b2f-a782-4521-b80f-f4ee194af653" volumeName="kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-tmp" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996365 5070 factory.go:223] Registration of the crio container factory successfully Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996385 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30e186ca-854c-417a-96ed-3a660db8b8f4" volumeName="kubernetes.io/secret/30e186ca-854c-417a-96ed-3a660db8b8f4-serving-cert" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996398 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56a08134-6094-4349-b252-ccf6d0aa8774" volumeName="kubernetes.io/projected/56a08134-6094-4349-b252-ccf6d0aa8774-kube-api-access-bzw9c" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996410 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79d5b339-6665-4fbe-92db-77fd202119d2" volumeName="kubernetes.io/empty-dir/79d5b339-6665-4fbe-92db-77fd202119d2-tmp-dir" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996422 5070 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996427 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996441 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faa532be-e39c-4a3d-bf81-da68387b9703" volumeName="kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996446 5070 factory.go:103] Registering Raw factory Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996458 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06297482-f53e-497f-8012-1c6a55b1462b" volumeName="kubernetes.io/projected/06297482-f53e-497f-8012-1c6a55b1462b-kube-api-access-qdzm6" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996463 5070 manager.go:1199] Started watching for new ooms in manager Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996471 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996538 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9f0e7740-17a0-460e-9407-5650fa708046" volumeName="kubernetes.io/projected/9f0e7740-17a0-460e-9407-5650fa708046-kube-api-access-h5scv" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996569 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b60126a0-255e-4b6d-9166-18c92777ddc5" volumeName="kubernetes.io/projected/b60126a0-255e-4b6d-9166-18c92777ddc5-kube-api-access-hgmhk" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996591 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf9ba972-af6a-4cda-a59b-f51874d49aae" volumeName="kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-daemon-config" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996606 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf9ba972-af6a-4cda-a59b-f51874d49aae" volumeName="kubernetes.io/projected/cf9ba972-af6a-4cda-a59b-f51874d49aae-kube-api-access-dw8z6" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996625 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" volumeName="kubernetes.io/projected/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-kube-api-access-zt78g" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996639 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11d10c3e-223f-41ef-ac3f-cd428bf43ebd" volumeName="kubernetes.io/configmap/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-serviceca" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996668 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47df06d1-8c08-47c2-9e1f-10636f6bf035" volumeName="kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-script-lib" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996689 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79d5b339-6665-4fbe-92db-77fd202119d2" volumeName="kubernetes.io/configmap/79d5b339-6665-4fbe-92db-77fd202119d2-config-volume" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996702 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a04618ce-f5a4-49ae-8fee-ffb082167b8c" volumeName="kubernetes.io/projected/a04618ce-f5a4-49ae-8fee-ffb082167b8c-kube-api-access-xhgb4" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996722 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13093b2f-a782-4521-b80f-f4ee194af653" volumeName="kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-snapshots" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996736 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13093b2f-a782-4521-b80f-f4ee194af653" volumeName="kubernetes.io/projected/13093b2f-a782-4521-b80f-f4ee194af653-kube-api-access-5l8qg" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996750 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13093b2f-a782-4521-b80f-f4ee194af653" volumeName="kubernetes.io/secret/13093b2f-a782-4521-b80f-f4ee194af653-serving-cert" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996776 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1adf30d9-26c7-46b0-b9bb-dfb96ba3711e" volumeName="kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-stats-auth" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996790 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="399ea207-fdb0-4a0a-a841-b113cffc4c3e" volumeName="kubernetes.io/configmap/399ea207-fdb0-4a0a-a841-b113cffc4c3e-iptables-alerter-script" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996809 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47df06d1-8c08-47c2-9e1f-10636f6bf035" volumeName="kubernetes.io/secret/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996821 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79d5b339-6665-4fbe-92db-77fd202119d2" volumeName="kubernetes.io/projected/79d5b339-6665-4fbe-92db-77fd202119d2-kube-api-access-tw4m7" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996833 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b60126a0-255e-4b6d-9166-18c92777ddc5" volumeName="kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-tmp" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996849 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1adf30d9-26c7-46b0-b9bb-dfb96ba3711e" volumeName="kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-default-certificate" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996864 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="463c6228-727e-4d4f-9b80-1db615657b4f" volumeName="kubernetes.io/projected/463c6228-727e-4d4f-9b80-1db615657b4f-kube-api-access-fd977" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996884 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47df06d1-8c08-47c2-9e1f-10636f6bf035" volumeName="kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-config" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996897 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68a3d167-dbc7-4155-aee3-e056b558ab8a" volumeName="kubernetes.io/projected/68a3d167-dbc7-4155-aee3-e056b558ab8a-kube-api-access-frs5z" seLinuxMountContext="" Oct 30 18:31:22.996913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996932 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996953 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4ec318-a75a-4dd0-9da0-ffc41310b641" volumeName="kubernetes.io/configmap/8b4ec318-a75a-4dd0-9da0-ffc41310b641-konnectivity-ca" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996964 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996980 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbf77bfc-9322-4635-9080-36631d7613df" volumeName="kubernetes.io/projected/cbf77bfc-9322-4635-9080-36631d7613df-kube-api-access-fmcdg" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996998 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19c4a917-c2ba-46e3-8b7f-d528b1c26bc9" volumeName="kubernetes.io/configmap/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-config" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997009 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f3acf05-0627-4917-b471-c27745e87bf0" volumeName="kubernetes.io/secret/1f3acf05-0627-4917-b471-c27745e87bf0-signing-key" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997026 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="56a08134-6094-4349-b252-ccf6d0aa8774" volumeName="kubernetes.io/configmap/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-config" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997038 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" volumeName="kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-binary-copy" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997050 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4ec318-a75a-4dd0-9da0-ffc41310b641" volumeName="kubernetes.io/secret/8b4ec318-a75a-4dd0-9da0-ffc41310b641-agent-certs" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997067 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997078 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b60126a0-255e-4b6d-9166-18c92777ddc5" volumeName="kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-tuned" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997095 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" volumeName="kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-trusted-ca" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997106 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19c4a917-c2ba-46e3-8b7f-d528b1c26bc9" volumeName="kubernetes.io/secret/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-serving-cert" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997123 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fdf2c77-42d8-4686-a656-334557d55531" volumeName="kubernetes.io/configmap/2fdf2c77-42d8-4686-a656-334557d55531-nginx-conf" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997134 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43" volumeName="kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997166 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68a3d167-dbc7-4155-aee3-e056b558ab8a" volumeName="kubernetes.io/secret/68a3d167-dbc7-4155-aee3-e056b558ab8a-cert" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997183 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85a0105a-b96c-449d-9e91-bd8e457c0a0a" volumeName="kubernetes.io/projected/85a0105a-b96c-449d-9e91-bd8e457c0a0a-kube-api-access-9fdkt" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997195 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf9ba972-af6a-4cda-a59b-f51874d49aae" volumeName="kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-cni-binary-copy" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997210 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" volumeName="kubernetes.io/secret/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-serving-cert" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997220 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faa532be-e39c-4a3d-bf81-da68387b9703" volumeName="kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997234 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13093b2f-a782-4521-b80f-f4ee194af653" volumeName="kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-service-ca-bundle" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997251 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997262 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="399ea207-fdb0-4a0a-a841-b113cffc4c3e" volumeName="kubernetes.io/projected/399ea207-fdb0-4a0a-a841-b113cffc4c3e-kube-api-access-fd8rj" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997277 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="729c533b-1000-4d81-ad6e-a06a15679e78" volumeName="kubernetes.io/projected/729c533b-1000-4d81-ad6e-a06a15679e78-kube-api-access-qd8pj" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997291 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72c54cf4-7294-4d5b-a0f5-61d989cd960f" volumeName="kubernetes.io/projected/72c54cf4-7294-4d5b-a0f5-61d989cd960f-kube-api-access-652nf" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997303 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="865d20d3-b981-4d85-a56b-4951ae770312" volumeName="kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997319 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e6926c1-1692-4111-85b6-319433b9bdfe" volumeName="kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997331 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="faa532be-e39c-4a3d-bf81-da68387b9703" volumeName="kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997349 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13093b2f-a782-4521-b80f-f4ee194af653" volumeName="kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997362 5070 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1adf30d9-26c7-46b0-b9bb-dfb96ba3711e" volumeName="kubernetes.io/projected/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-kube-api-access-zpkgn" seLinuxMountContext="" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997373 5070 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.997381 5070 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:31:22.998606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:22.996901 5070 manager.go:319] Starting recovery of all containers Oct 30 18:31:23.015353 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.015329 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:31:23.015589 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.015565 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:31:23.021656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.021637 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:31:23.021735 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.021679 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:31:23.022697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.022647 5070 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:31:23.024431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.024129 5070 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:31:23.024431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.024275 5070 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:31:23.024431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.024329 5070 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:31:23.024431 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:23.024374 5070 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:31:23.040529 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.040500 5070 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:31:23.040654 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.040566 5070 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 18:31:23.043726 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.043701 5070 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="778b346007c5482006fc6df4052d835949292287c9f7fd98e59aeb86d29676ba" exitCode=0 Oct 30 18:31:23.043726 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.043721 5070 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="3bb7caf48f826015d2d712bc2cd31b40b60520b917b27bdd91fac0d7bc79b908" exitCode=0 Oct 30 18:31:23.043726 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.043729 5070 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="755070d4d2723437f017a7022b4941c025c1df8c5cbc8cfe501e9764cb2dc8d3" exitCode=0 Oct 30 18:31:23.043935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.043735 5070 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="7b2aabf97f5284711fdc734d8254d77495295ad980c8a38656512cbc0f793293" exitCode=0 Oct 30 18:31:23.043935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.043741 5070 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="c333f72351781863bdf6926ad8d8992cdd0111623d1c1035a0117858db66e70f" exitCode=0 Oct 30 18:31:23.043935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.043746 5070 generic.go:356] "Generic (PLEG): container finished" podID="739cc6ce-77da-492c-9cfc-6bfe5c50dd56" containerID="1801c2eb8f7de05caa7bba8022648212b1c611167a2936d555e1416568c3e887" exitCode=0 Oct 30 18:31:23.047427 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.047411 5070 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:31:23.060632 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.060604 5070 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:31:23.060740 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.060639 5070 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:31:23.070741 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.070722 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:31:23.071065 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.071041 5070 generic.go:356] "Generic (PLEG): container finished" podID="47df06d1-8c08-47c2-9e1f-10636f6bf035" containerID="dbf87f3fac62d960281e4e3abbff71574f47f8e79d8d8350ebd70fb97fac956a" exitCode=1 Oct 30 18:31:23.078690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.078654 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:31:23.078799 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.078688 5070 generic.go:356] "Generic (PLEG): container finished" podID="eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3" containerID="4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55" exitCode=255 Oct 30 18:31:23.084894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.084822 5070 generic.go:356] "Generic (PLEG): container finished" podID="09d214f128413268df5e715b75027209" containerID="2ea2aea6331069a08c5e5813d6a3ce9bc720d5b4b518b754c7be824f2a525b2a" exitCode=0 Oct 30 18:31:23.103695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.103676 5070 manager.go:324] Recovery completed Oct 30 18:31:23.126321 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:23.126269 5070 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 18:31:23.127465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127444 5070 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:31:23.127465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127463 5070 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:31:23.127652 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127486 5070 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:31:23.127717 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127697 5070 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 18:31:23.127775 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127721 5070 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 18:31:23.127775 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127749 5070 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 18:31:23.127775 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127758 5070 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 18:31:23.127775 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127766 5070 policy_none.go:50] "Start" Oct 30 18:31:23.127956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127788 5070 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:31:23.127956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.127804 5070 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:31:23.128071 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.128058 5070 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 18:31:23.128123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.128075 5070 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 18:31:23.128123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.128083 5070 policy_none.go:44] "Start" Oct 30 18:31:23.133167 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.133118 5070 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:31:23.137599 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.137579 5070 manager.go:341] "Starting Device Plugin manager" Oct 30 18:31:23.138051 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:23.137605 5070 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:31:23.138105 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138053 5070 server.go:87] "Starting device plugin registration server" Oct 30 18:31:23.138318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138308 5070 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:31:23.138353 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138320 5070 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:31:23.138459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138443 5070 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:31:23.138587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138556 5070 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:31:23.138653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138613 5070 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:31:23.138653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138620 5070 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:31:23.138787 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.138685 5070 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:31:23.13858457Z","UUID":"c4d1f724-34b7-4849-8d74-cad5d34181c2","Handler":null,"Name":"","Endpoint":""} Oct 30 18:31:23.139959 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.139937 5070 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:31:23.140028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.139994 5070 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:31:23.140243 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:23.140223 5070 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:31:23.238683 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.238656 5070 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:31:23.240026 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.240008 5070 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientMemory" Oct 30 18:31:23.240114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.240039 5070 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 18:31:23.240114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.240055 5070 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-102.ec2.internal" event="NodeHasSufficientPID" Oct 30 18:31:23.240114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.240106 5070 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.282979 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.282953 5070 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.283081 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.283029 5070 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.327040 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.326989 5070 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal"] Oct 30 18:31:23.327690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327673 5070 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.327733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327688 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" event={"ID":"bdb7686e846fff2d163fb7282f45a982","Type":"ContainerStarted","Data":"f976e0453010ee6679814d94c6a49218e44fa33585f29da70a7c50fe56c0b8b0"} Oct 30 18:31:23.327733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327726 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" event={"ID":"bdb7686e846fff2d163fb7282f45a982","Type":"ContainerStarted","Data":"cd57ad319a5ed651955149203d269e2fe9eb8527bf6f769fd070e9435003b189"} Oct 30 18:31:23.327792 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327740 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" event={"ID":"09d214f128413268df5e715b75027209","Type":"ContainerStarted","Data":"fb0acbf816f067bf113461e36709c6720894f3c9bbec9d6bf2ff33b586b78375"} Oct 30 18:31:23.327792 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327752 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" event={"ID":"09d214f128413268df5e715b75027209","Type":"ContainerDied","Data":"2ea2aea6331069a08c5e5813d6a3ce9bc720d5b4b518b754c7be824f2a525b2a"} Oct 30 18:31:23.327792 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327765 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" event={"ID":"09d214f128413268df5e715b75027209","Type":"ContainerStarted","Data":"13d5ed9a8be62edff37c46a15814e007cc1707c525c900794735bb85ae889726"} Oct 30 18:31:23.327915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.327899 5070 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.355867 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.355817 5070 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:31:23.355867 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:23.355865 5070 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.356527 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.356511 5070 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:31:23.356603 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:23.356550 5070 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.442350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.442315 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.442350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.442350 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdb7686e846fff2d163fb7282f45a982-config\") pod \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" (UID: \"bdb7686e846fff2d163fb7282f45a982\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.442520 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.442370 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.543357 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.543329 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.543524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.543362 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.543524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.543386 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdb7686e846fff2d163fb7282f45a982-config\") pod \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" (UID: \"bdb7686e846fff2d163fb7282f45a982\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.543524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.543427 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.543524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.543454 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d214f128413268df5e715b75027209-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" (UID: \"09d214f128413268df5e715b75027209\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.543524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.543427 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bdb7686e846fff2d163fb7282f45a982-config\") pod \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" (UID: \"bdb7686e846fff2d163fb7282f45a982\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:23.976003 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:23.975968 5070 apiserver.go:52] "Watching apiserver" Oct 30 18:31:24.058894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.058863 5070 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:31:24.059567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.059540 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n922x","openshift-dns/node-resolver-sm8gj","openshift-multus/multus-fxrmc","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc","kube-system/global-pull-secret-syncer-jw7nd","openshift-cluster-node-tuning-operator/tuned-7lgct","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz","openshift-insights/insights-operator-7c4986d449-zkkkh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal","openshift-network-console/networking-console-plugin-86c478bb88-glvc2","openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9","openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5","openshift-ingress/router-default-75fdfd5b99-vpjd7","openshift-service-ca/service-ca-5fffc97bdf-9q4dn","openshift-image-registry/image-registry-6dd54f845-4vwhq","openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc","openshift-network-operator/iptables-alerter-l5zgj","openshift-insights/insights-runtime-extractor-srvzm","openshift-insights/periodic-gathering-v955r-lpqmq","openshift-image-registry/node-ca-9nr4w","openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf","openshift-dns/dns-default-lpq29","kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal","kube-system/kas-connection-checker-685895c7c9-9q5g7","kube-system/kas-connection-checker-685895c7c9-twcqb","openshift-multus/multus-additional-cni-plugins-jjvbp","kube-system/kas-connection-checker-685895c7c9-xzmd9","openshift-image-registry/image-registry-687ccc8fd4-p2sn5","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8","openshift-multus/network-metrics-daemon-xlm58","openshift-ingress-canary/ingress-canary-jr9ld","kube-system/konnectivity-agent-plcdm","openshift-network-diagnostics/network-check-target-gwzzt","openshift-console-operator/console-operator-7cd56b8dbc-c262b","openshift-ovn-kubernetes/ovnkube-node-l6tz8"] Oct 30 18:31:24.061188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.061166 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.061509 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.061480 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.061871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.061854 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.062438 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.062421 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.066978 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.066952 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:24.067064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.066992 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.072042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.072023 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.074695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.074676 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.075222 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.075207 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.076995 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.076980 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.077233 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077215 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.077497 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077471 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 18:31:24.077538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077482 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 18:31:24.077538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077528 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 18:31:24.077603 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077531 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 18:31:24.077703 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077688 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.077818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077793 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.077865 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.077851 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 18:31:24.078137 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.078100 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 18:31:24.078286 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.078272 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.078346 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.078303 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.079488 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.079469 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 18:31:24.090421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.090396 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 18:31:24.090621 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.090597 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.091749 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.091729 5070 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:24.092312 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.092295 5070 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:24.095020 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.095002 5070 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:31:24.100654 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.100635 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:31:24.101441 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101419 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101421 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101473 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101485 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101501 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101515 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101524 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101506 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.101595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101553 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.101946 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101733 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.101946 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.101773 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.103320 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103306 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 18:31:24.103460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103349 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 18:31:24.103460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103414 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 18:31:24.103460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103416 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 18:31:24.103460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103436 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 18:31:24.103460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103449 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.103460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103458 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 18:31:24.103926 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103449 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 18:31:24.103926 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103922 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:31:24.104030 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103968 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.104030 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.103999 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.104030 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104018 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:31:24.104271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104030 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.104271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104064 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.104271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104075 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wkxtp\"" Oct 30 18:31:24.104271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104077 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 18:31:24.104271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104126 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:31:24.104271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104193 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-bqvz6\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104297 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gbkqj\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104321 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104300 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104325 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xfxj4\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104332 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104368 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8srjr\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104390 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-56ppb\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104480 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-k7jl9\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104493 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ft7c\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104498 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-r8p5q\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104522 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-gjg7l\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104536 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tvrn8\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104550 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtqtd\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104578 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104592 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kkmm\"" Oct 30 18:31:24.104602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104601 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104661 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104679 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-ghgqv\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104688 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104695 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jtzwt\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104683 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104722 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wrnjz\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104730 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-shvwg\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104809 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bz2z9\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104845 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104865 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104866 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104878 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pvt2z\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104890 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zcslk\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104903 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsmgf\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104902 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104939 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:31:24.105373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.104943 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.113257 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.113241 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.114355 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.114339 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:31:24.115422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115369 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 18:31:24.115510 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115486 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115510 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115490 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:31:24.115678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115508 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115576 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.115678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115593 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.115678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115605 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115598 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115713 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115725 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115764 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115767 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115781 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115810 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115814 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115791 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115827 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115767 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115842 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.115890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115867 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115919 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115966 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.115998 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.116031 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.116064 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.116222 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:31:24.116334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.116223 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:31:24.118394 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.118381 5070 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:31:24.118480 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.118421 5070 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-102.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-102.ec2.internal" Oct 30 18:31:24.119040 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.119028 5070 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:31:24.119073 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.119059 5070 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-102.ec2.internal" Oct 30 18:31:24.146906 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.146881 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-kubelet\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.146993 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.146909 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-var-lib-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.146993 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.146932 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-log-socket\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.146993 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.146960 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-default-certificate\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.147132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147000 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-tuned\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.147132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147024 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.147132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147043 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmcdg\" (UniqueName: \"kubernetes.io/projected/cbf77bfc-9322-4635-9080-36631d7613df-kube-api-access-fmcdg\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.147132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147097 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwfcw\" (UniqueName: \"kubernetes.io/projected/30e186ca-854c-417a-96ed-3a660db8b8f4-kube-api-access-xwfcw\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:24.147132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147103 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-tuned\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147129 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-netd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147186 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-sys-fs\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147254 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd977\" (UniqueName: \"kubernetes.io/projected/463c6228-727e-4d4f-9b80-1db615657b4f-kube-api-access-fd977\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147254 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-default-certificate\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147298 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147338 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147368 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.147400 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147398 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-env-overrides\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.147743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147446 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-lib-modules\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.147743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147540 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79d5b339-6665-4fbe-92db-77fd202119d2-tmp-dir\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.147743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147653 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw8z6\" (UniqueName: \"kubernetes.io/projected/cf9ba972-af6a-4cda-a59b-f51874d49aae-kube-api-access-dw8z6\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.147743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147671 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79d5b339-6665-4fbe-92db-77fd202119d2-tmp-dir\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.147743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147687 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcbsv\" (UniqueName: \"kubernetes.io/projected/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-kube-api-access-dcbsv\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:24.147743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147725 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-original-pull-secret\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147746 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-k8s-cni-cncf-io\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147772 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-netns\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147788 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-env-overrides\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147797 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87hnt\" (UniqueName: \"kubernetes.io/projected/4bb762ff-7409-4218-8421-d0b376a46f98-kube-api-access-87hnt\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147822 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec318-a75a-4dd0-9da0-ffc41310b641-agent-certs\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.147969 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7rmv\" (UniqueName: \"kubernetes.io/projected/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-kube-api-access-f7rmv\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.148019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148009 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148037 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148064 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148108 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148135 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148173 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148201 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148208 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b4ec318-a75a-4dd0-9da0-ffc41310b641-agent-certs\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148224 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-var-lib-kubelet\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148262 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-systemd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148308 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-conf\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.148379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148340 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhgb4\" (UniqueName: \"kubernetes.io/projected/a04618ce-f5a4-49ae-8fee-ffb082167b8c-kube-api-access-xhgb4\") pod \"kas-connection-checker-685895c7c9-xzmd9\" (UID: \"a04618ce-f5a4-49ae-8fee-ffb082167b8c\") " pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148392 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148405 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-system-cni-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148440 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf77bfc-9322-4635-9080-36631d7613df-data-volume\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148465 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148491 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-kubelet-config\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148552 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cbf77bfc-9322-4635-9080-36631d7613df-data-volume\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148561 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148594 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148618 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-tmp\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148647 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-multus\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148685 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148710 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvpks\" (UniqueName: \"kubernetes.io/projected/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-kube-api-access-fvpks\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148735 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1f3acf05-0627-4917-b471-c27745e87bf0-signing-key\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148763 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b60126a0-255e-4b6d-9166-18c92777ddc5-tmp\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148769 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148776 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-serving-cert\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148787 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148815 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.148901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148834 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148917 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd8pj\" (UniqueName: \"kubernetes.io/projected/729c533b-1000-4d81-ad6e-a06a15679e78-kube-api-access-qd8pj\") pod \"migrator-66df5d496b-zvks5\" (UID: \"729c533b-1000-4d81-ad6e-a06a15679e78\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.148966 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-etc-kubernetes\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149009 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1f3acf05-0627-4917-b471-c27745e87bf0-signing-key\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149074 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-config\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149105 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-system-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149163 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-serving-cert\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149198 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frs5z\" (UniqueName: \"kubernetes.io/projected/68a3d167-dbc7-4155-aee3-e056b558ab8a-kube-api-access-frs5z\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149218 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-ovn\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149233 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-stats-auth\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149250 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30e186ca-854c-417a-96ed-3a660db8b8f4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149254 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-config\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149364 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149399 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-serving-cert\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149438 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-host\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149463 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/30e186ca-854c-417a-96ed-3a660db8b8f4-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149483 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-stats-auth\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149472 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149562 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149593 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-cnibin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149618 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-kubelet\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.149643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149646 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149695 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-slash\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149717 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149734 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/399ea207-fdb0-4a0a-a841-b113cffc4c3e-iptables-alerter-script\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149778 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5scv\" (UniqueName: \"kubernetes.io/projected/9f0e7740-17a0-460e-9407-5650fa708046-kube-api-access-h5scv\") pod \"volume-data-source-validator-58c9f5f46d-c4nbz\" (UID: \"9f0e7740-17a0-460e-9407-5650fa708046\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149807 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-run\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149830 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-sys\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149855 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4m7\" (UniqueName: \"kubernetes.io/projected/79d5b339-6665-4fbe-92db-77fd202119d2-kube-api-access-tw4m7\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149872 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-serving-cert\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149879 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-dbus\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149871 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149905 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149904 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149919 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149946 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd8rj\" (UniqueName: \"kubernetes.io/projected/399ea207-fdb0-4a0a-a841-b113cffc4c3e-kube-api-access-fd8rj\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.149962 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-daemon-config\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150002 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652nf\" (UniqueName: \"kubernetes.io/projected/72c54cf4-7294-4d5b-a0f5-61d989cd960f-kube-api-access-652nf\") pod \"network-check-source-6f78d59f99-xjnjf\" (UID: \"72c54cf4-7294-4d5b-a0f5-61d989cd960f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150005 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/399ea207-fdb0-4a0a-a841-b113cffc4c3e-iptables-alerter-script\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150052 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt78g\" (UniqueName: \"kubernetes.io/projected/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-kube-api-access-zt78g\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150166 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-netns\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150235 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-daemon-config\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150244 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-registration-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150294 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/399ea207-fdb0-4a0a-a841-b113cffc4c3e-host-slash\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150338 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.150371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150383 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-script-lib\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150492 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgmhk\" (UniqueName: \"kubernetes.io/projected/b60126a0-255e-4b6d-9166-18c92777ddc5-kube-api-access-hgmhk\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150552 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150558 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-script-lib\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150629 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150766 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150814 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150817 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zshw\" (UniqueName: \"kubernetes.io/projected/10ed4e5d-002c-48ca-ad1e-199694eef36b-kube-api-access-5zshw\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150971 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-host\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.150999 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1f3acf05-0627-4917-b471-c27745e87bf0-signing-cabundle\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151029 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2fdf2c77-42d8-4686-a656-334557d55531-nginx-conf\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151111 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-kubernetes\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151186 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79d5b339-6665-4fbe-92db-77fd202119d2-config-volume\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151205 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2fdf2c77-42d8-4686-a656-334557d55531-nginx-conf\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:24.151213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151206 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151223 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1f3acf05-0627-4917-b471-c27745e87bf0-signing-cabundle\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151231 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-service-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151283 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-node-log\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151310 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-etc-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151348 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-serviceca\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151380 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-hostroot\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151398 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-conf-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151410 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79d5b339-6665-4fbe-92db-77fd202119d2-config-volume\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151437 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151533 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-bin\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151571 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzw9c\" (UniqueName: \"kubernetes.io/projected/56a08134-6094-4349-b252-ccf6d0aa8774-kube-api-access-bzw9c\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151578 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-serviceca\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151600 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-config\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151626 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-modprobe-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151627 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151662 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysconfig\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151699 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30e186ca-854c-417a-96ed-3a660db8b8f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151702 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13093b2f-a782-4521-b80f-f4ee194af653-service-ca-bundle\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151741 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151773 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-config\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151779 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13093b2f-a782-4521-b80f-f4ee194af653-serving-cert\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151799 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/30e186ca-854c-417a-96ed-3a660db8b8f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151830 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151864 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-socket-dir-parent\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151944 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13093b2f-a782-4521-b80f-f4ee194af653-serving-cert\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151962 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a3d167-dbc7-4155-aee3-e056b558ab8a-cert\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.151980 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-trusted-ca\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152018 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152061 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152125 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152178 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-config\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152216 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovn-node-metrics-cert\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152247 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-device-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152249 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68a3d167-dbc7-4155-aee3-e056b558ab8a-cert\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152295 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf77bfc-9322-4635-9080-36631d7613df-crio-socket\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152343 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdzm6\" (UniqueName: \"kubernetes.io/projected/06297482-f53e-497f-8012-1c6a55b1462b-kube-api-access-qdzm6\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152373 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovnkube-config\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152392 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcsbm\" (UniqueName: \"kubernetes.io/projected/9c695b41-2900-4b0d-832b-bb35456eb816-kube-api-access-mcsbm\") pod \"kas-connection-checker-685895c7c9-twcqb\" (UID: \"9c695b41-2900-4b0d-832b-bb35456eb816\") " pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152470 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b4ec318-a75a-4dd0-9da0-ffc41310b641-konnectivity-ca\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152504 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463c6228-727e-4d4f-9b80-1db615657b4f-tmp-dir\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152533 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152561 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s49v\" (UniqueName: \"kubernetes.io/projected/1f3acf05-0627-4917-b471-c27745e87bf0-kube-api-access-2s49v\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152573 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/47df06d1-8c08-47c2-9e1f-10636f6bf035-ovn-node-metrics-cert\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152586 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68rjt\" (UniqueName: \"kubernetes.io/projected/0fae4e48-2851-4c67-9615-ad86f9795397-kube-api-access-68rjt\") pod \"kas-connection-checker-685895c7c9-9q5g7\" (UID: \"0fae4e48-2851-4c67-9615-ad86f9795397\") " pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152607 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/463c6228-727e-4d4f-9b80-1db615657b4f-tmp-dir\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152613 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-snapshots\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152643 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152676 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-systemd-units\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152680 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b4ec318-a75a-4dd0-9da0-ffc41310b641-konnectivity-ca\") pod \"konnectivity-agent-plcdm\" (UID: \"8b4ec318-a75a-4dd0-9da0-ffc41310b641\") " pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152710 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152730 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-snapshots\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152740 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8qg\" (UniqueName: \"kubernetes.io/projected/13093b2f-a782-4521-b80f-f4ee194af653-kube-api-access-5l8qg\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152730 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-trusted-ca\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152762 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152777 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-etc-selinux\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152782 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152807 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fdkt\" (UniqueName: \"kubernetes.io/projected/85a0105a-b96c-449d-9e91-bd8e457c0a0a-kube-api-access-9fdkt\") pod \"migrator-66df5d496b-ffzcc\" (UID: \"85a0105a-b96c-449d-9e91-bd8e457c0a0a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152935 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152991 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.152993 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153031 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cnibin\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153123 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153160 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153194 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153217 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpkgn\" (UniqueName: \"kubernetes.io/projected/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-kube-api-access-zpkgn\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153236 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153251 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-tmp\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153278 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153290 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153329 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13093b2f-a782-4521-b80f-f4ee194af653-tmp\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153426 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cni-binary-copy\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153527 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/463c6228-727e-4d4f-9b80-1db615657b4f-hosts-file\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153569 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-cni-binary-copy\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153591 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-bin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153631 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zhgq\" (UniqueName: \"kubernetes.io/projected/47df06d1-8c08-47c2-9e1f-10636f6bf035-kube-api-access-8zhgq\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153651 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-systemd\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153672 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153703 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-multus-certs\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153720 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/06297482-f53e-497f-8012-1c6a55b1462b-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153738 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-socket-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.153752 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.154200 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.154391 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf9ba972-af6a-4cda-a59b-f51874d49aae-cni-binary-copy\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.154398 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.154413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.154461 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/06297482-f53e-497f-8012-1c6a55b1462b-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.180666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.180638 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzw9c\" (UniqueName: \"kubernetes.io/projected/56a08134-6094-4349-b252-ccf6d0aa8774-kube-api-access-bzw9c\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.181117 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.181086 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5scv\" (UniqueName: \"kubernetes.io/projected/9f0e7740-17a0-460e-9407-5650fa708046-kube-api-access-h5scv\") pod \"volume-data-source-validator-58c9f5f46d-c4nbz\" (UID: \"9f0e7740-17a0-460e-9407-5650fa708046\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-c4nbz" Oct 30 18:31:24.181389 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.181365 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhgb4\" (UniqueName: \"kubernetes.io/projected/a04618ce-f5a4-49ae-8fee-ffb082167b8c-kube-api-access-xhgb4\") pod \"kas-connection-checker-685895c7c9-xzmd9\" (UID: \"a04618ce-f5a4-49ae-8fee-ffb082167b8c\") " pod="kube-system/kas-connection-checker-685895c7c9-xzmd9" Oct 30 18:31:24.181522 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.181505 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcsbm\" (UniqueName: \"kubernetes.io/projected/9c695b41-2900-4b0d-832b-bb35456eb816-kube-api-access-mcsbm\") pod \"kas-connection-checker-685895c7c9-twcqb\" (UID: \"9c695b41-2900-4b0d-832b-bb35456eb816\") " pod="kube-system/kas-connection-checker-685895c7c9-twcqb" Oct 30 18:31:24.188084 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.188056 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmcdg\" (UniqueName: \"kubernetes.io/projected/cbf77bfc-9322-4635-9080-36631d7613df-kube-api-access-fmcdg\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.188201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.188162 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87hnt\" (UniqueName: \"kubernetes.io/projected/4bb762ff-7409-4218-8421-d0b376a46f98-kube-api-access-87hnt\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.188201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.188184 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68rjt\" (UniqueName: \"kubernetes.io/projected/0fae4e48-2851-4c67-9615-ad86f9795397-kube-api-access-68rjt\") pod \"kas-connection-checker-685895c7c9-9q5g7\" (UID: \"0fae4e48-2851-4c67-9615-ad86f9795397\") " pod="kube-system/kas-connection-checker-685895c7c9-9q5g7" Oct 30 18:31:24.188274 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.188237 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvpks\" (UniqueName: \"kubernetes.io/projected/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-kube-api-access-fvpks\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.193181 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.193160 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdzm6\" (UniqueName: \"kubernetes.io/projected/06297482-f53e-497f-8012-1c6a55b1462b-kube-api-access-qdzm6\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.193257 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.193229 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s49v\" (UniqueName: \"kubernetes.io/projected/1f3acf05-0627-4917-b471-c27745e87bf0-kube-api-access-2s49v\") pod \"service-ca-5fffc97bdf-9q4dn\" (UID: \"1f3acf05-0627-4917-b471-c27745e87bf0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-9q4dn" Oct 30 18:31:24.193640 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.193619 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgmhk\" (UniqueName: \"kubernetes.io/projected/b60126a0-255e-4b6d-9166-18c92777ddc5-kube-api-access-hgmhk\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.193929 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.193914 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.198236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.198215 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8qg\" (UniqueName: \"kubernetes.io/projected/13093b2f-a782-4521-b80f-f4ee194af653-kube-api-access-5l8qg\") pod \"insights-operator-7c4986d449-zkkkh\" (UID: \"13093b2f-a782-4521-b80f-f4ee194af653\") " pod="openshift-insights/insights-operator-7c4986d449-zkkkh" Oct 30 18:31:24.198675 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.198655 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.198811 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.198791 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zhgq\" (UniqueName: \"kubernetes.io/projected/47df06d1-8c08-47c2-9e1f-10636f6bf035-kube-api-access-8zhgq\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.198917 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.198900 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-652nf\" (UniqueName: \"kubernetes.io/projected/72c54cf4-7294-4d5b-a0f5-61d989cd960f-kube-api-access-652nf\") pod \"network-check-source-6f78d59f99-xjnjf\" (UID: \"72c54cf4-7294-4d5b-a0f5-61d989cd960f\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xjnjf" Oct 30 18:31:24.200551 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.200533 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4m7\" (UniqueName: \"kubernetes.io/projected/79d5b339-6665-4fbe-92db-77fd202119d2-kube-api-access-tw4m7\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.202666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.202646 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7rmv\" (UniqueName: \"kubernetes.io/projected/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-kube-api-access-f7rmv\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.202747 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.202678 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fdkt\" (UniqueName: \"kubernetes.io/projected/85a0105a-b96c-449d-9e91-bd8e457c0a0a-kube-api-access-9fdkt\") pod \"migrator-66df5d496b-ffzcc\" (UID: \"85a0105a-b96c-449d-9e91-bd8e457c0a0a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ffzcc" Oct 30 18:31:24.203342 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.203326 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qqs\" (UniqueName: \"kubernetes.io/projected/4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43-kube-api-access-j6qqs\") pod \"network-check-target-gwzzt\" (UID: \"4e1513cf-a8c5-478a-9cb9-2e99f2f2ef43\") " pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:24.203393 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.203356 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.203850 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.203826 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.204243 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.204224 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcbsv\" (UniqueName: \"kubernetes.io/projected/19c4a917-c2ba-46e3-8b7f-d528b1c26bc9-kube-api-access-dcbsv\") pod \"service-ca-operator-544d749c8c-vtvc9\" (UID: \"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" Oct 30 18:31:24.204314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.204284 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw8z6\" (UniqueName: \"kubernetes.io/projected/cf9ba972-af6a-4cda-a59b-f51874d49aae-kube-api-access-dw8z6\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.204554 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.204537 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd8pj\" (UniqueName: \"kubernetes.io/projected/729c533b-1000-4d81-ad6e-a06a15679e78-kube-api-access-qd8pj\") pod \"migrator-66df5d496b-zvks5\" (UID: \"729c533b-1000-4d81-ad6e-a06a15679e78\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zvks5" Oct 30 18:31:24.204731 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.204714 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") pod \"periodic-gathering-v955r-lpqmq\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:31:24.205907 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.205890 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd977\" (UniqueName: \"kubernetes.io/projected/463c6228-727e-4d4f-9b80-1db615657b4f-kube-api-access-fd977\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.206378 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.206359 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt78g\" (UniqueName: \"kubernetes.io/projected/eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3-kube-api-access-zt78g\") pod \"console-operator-7cd56b8dbc-c262b\" (UID: \"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:24.206415 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.206391 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpkgn\" (UniqueName: \"kubernetes.io/projected/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-kube-api-access-zpkgn\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.207172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.207156 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zshw\" (UniqueName: \"kubernetes.io/projected/10ed4e5d-002c-48ca-ad1e-199694eef36b-kube-api-access-5zshw\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.208977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.208956 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwfcw\" (UniqueName: \"kubernetes.io/projected/30e186ca-854c-417a-96ed-3a660db8b8f4-kube-api-access-xwfcw\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8\" (UID: \"30e186ca-854c-417a-96ed-3a660db8b8f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" Oct 30 18:31:24.209561 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.209545 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd8rj\" (UniqueName: \"kubernetes.io/projected/399ea207-fdb0-4a0a-a841-b113cffc4c3e-kube-api-access-fd8rj\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.215040 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.215020 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:24.215629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.215612 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frs5z\" (UniqueName: \"kubernetes.io/projected/68a3d167-dbc7-4155-aee3-e056b558ab8a-kube-api-access-frs5z\") pod \"ingress-canary-jr9ld\" (UID: \"68a3d167-dbc7-4155-aee3-e056b558ab8a\") " pod="openshift-ingress-canary/ingress-canary-jr9ld" Oct 30 18:31:24.254121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254065 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.254121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254103 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-etc-kubernetes\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254120 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-system-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254134 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-ovn\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254167 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254198 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-host\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254221 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254226 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-etc-kubernetes\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254236 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-ovn\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.254242 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.754219836 +0000 UTC m=+1.869353750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : configmap references non-existent config key: service-ca.crt Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254267 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-kubelet-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254290 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-system-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254294 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-cnibin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254313 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-host\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254322 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-kubelet\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254326 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254349 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-slash\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254357 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-cnibin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254372 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-run\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254381 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-kubelet\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254393 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-sys\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254415 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-dbus\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254428 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-run\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254394 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-slash\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254430 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254471 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-netns\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254489 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-dbus\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254486 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-sys\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254517 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-netns\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254540 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-registration-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254562 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/399ea207-fdb0-4a0a-a841-b113cffc4c3e-host-slash\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254591 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254597 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-registration-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254607 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/399ea207-fdb0-4a0a-a841-b113cffc4c3e-host-slash\") pod \"iptables-alerter-l5zgj\" (UID: \"399ea207-fdb0-4a0a-a841-b113cffc4c3e\") " pod="openshift-network-operator/iptables-alerter-l5zgj" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254626 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254632 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254645 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-host\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254667 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-kubernetes\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254682 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-node-log\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254696 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-etc-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254711 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-hostroot\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.254712 5070 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254725 5070 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254748 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-hostroot\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254751 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-kubernetes\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254749 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11d10c3e-223f-41ef-ac3f-cd428bf43ebd-host\") pod \"node-ca-9nr4w\" (UID: \"11d10c3e-223f-41ef-ac3f-cd428bf43ebd\") " pod="openshift-image-registry/node-ca-9nr4w" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.254773 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.754761146 +0000 UTC m=+1.869895080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254787 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-etc-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254789 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-conf-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254794 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-node-log\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254820 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-conf-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254823 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-bin\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254852 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-modprobe-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254873 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysconfig\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254876 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-bin\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.254871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254900 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-socket-dir-parent\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254923 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254951 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254957 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysconfig\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254963 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-socket-dir-parent\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254973 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-device-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254994 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254997 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-modprobe-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.254996 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf77bfc-9322-4635-9080-36631d7613df-crio-socket\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255037 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cbf77bfc-9322-4635-9080-36631d7613df-crio-socket\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255042 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-device-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255050 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255078 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-systemd-units\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255094 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255111 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-etc-selinux\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255140 5070 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255166 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-systemd-units\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255186 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-etc-selinux\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255194 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cnibin\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255222 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.755189111 +0000 UTC m=+1.870323005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255223 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-cnibin\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255237 5070 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255251 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255282 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.755266718 +0000 UTC m=+1.870400647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255306 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255314 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-multus-cni-dir\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255346 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255399 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/463c6228-727e-4d4f-9b80-1db615657b4f-hosts-file\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255427 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-bin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255441 5070 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255465 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-d\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255495 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-systemd\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255501 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.755487367 +0000 UTC m=+1.870621278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255450 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-systemd\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255497 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/463c6228-727e-4d4f-9b80-1db615657b4f-hosts-file\") pod \"node-resolver-sm8gj\" (UID: \"463c6228-727e-4d4f-9b80-1db615657b4f\") " pod="openshift-dns/node-resolver-sm8gj" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255507 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-bin\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255536 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-multus-certs\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255553 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-socket-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255563 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-multus-certs\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255568 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-kubelet\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255589 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-kubelet\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255594 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-var-lib-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255618 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-log-socket\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255620 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-socket-dir\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255644 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255660 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-var-lib-openvswitch\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255669 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-netd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255693 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-sys-fs\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255726 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-cni-netd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255725 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255759 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-lib-modules\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255771 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4bb762ff-7409-4218-8421-d0b376a46f98-sys-fs\") pod \"aws-ebs-csi-driver-node-88zjz\" (UID: \"4bb762ff-7409-4218-8421-d0b376a46f98\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-88zjz" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255783 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-k8s-cni-cncf-io\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255790 5070 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255809 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-netns\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255835 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.7558209 +0000 UTC m=+1.870954799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255844 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-netns\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255863 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255870 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-lib-modules\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255881 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-run-k8s-cni-cncf-io\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255686 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-log-socket\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255891 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255922 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255947 5070 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.255988 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:24.755975282 +0000 UTC m=+1.871109188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255948 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-var-lib-kubelet\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.255999 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-var-lib-kubelet\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256022 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-systemd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256039 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256048 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-conf\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256075 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/47df06d1-8c08-47c2-9e1f-10636f6bf035-run-systemd\") pod \"ovnkube-node-l6tz8\" (UID: \"47df06d1-8c08-47c2-9e1f-10636f6bf035\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256074 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-system-cni-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256108 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-kubelet-config\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256109 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/739cc6ce-77da-492c-9cfc-6bfe5c50dd56-system-cni-dir\") pod \"multus-additional-cni-plugins-jjvbp\" (UID: \"739cc6ce-77da-492c-9cfc-6bfe5c50dd56\") " pod="openshift-multus/multus-additional-cni-plugins-jjvbp" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256171 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-multus\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256200 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cbf77bfc-9322-4635-9080-36631d7613df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256236 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cf9ba972-af6a-4cda-a59b-f51874d49aae-host-var-lib-cni-multus\") pod \"multus-fxrmc\" (UID: \"cf9ba972-af6a-4cda-a59b-f51874d49aae\") " pod="openshift-multus/multus-fxrmc" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256258 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/deaa4bf3-e4f9-4b22-9bda-eba0080c0697-kubelet-config\") pod \"global-pull-secret-syncer-jw7nd\" (UID: \"deaa4bf3-e4f9-4b22-9bda-eba0080c0697\") " pod="kube-system/global-pull-secret-syncer-jw7nd" Oct 30 18:31:24.256724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.256259 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b60126a0-255e-4b6d-9166-18c92777ddc5-etc-sysctl-conf\") pod \"tuned-7lgct\" (UID: \"b60126a0-255e-4b6d-9166-18c92777ddc5\") " pod="openshift-cluster-node-tuning-operator/tuned-7lgct" Oct 30 18:31:24.258925 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.257246 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10ed4e5d-002c-48ca-ad1e-199694eef36b-metrics-certs\") pod \"network-metrics-daemon-xlm58\" (UID: \"10ed4e5d-002c-48ca-ad1e-199694eef36b\") " pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.258925 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.257477 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"image-registry-687ccc8fd4-p2sn5\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.258925 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.258224 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"image-registry-6dd54f845-4vwhq\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.361769 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.361739 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:24.363837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.363814 5070 scope.go:122] "RemoveContainer" containerID="4b5c07ff7d0107fb2b93da3a55b32de0e2e948e12a1cfb72f9de077138421b55" Oct 30 18:31:24.367536 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.367511 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xlm58" Oct 30 18:31:24.372975 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.372956 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:24.396586 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.396547 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:24.502053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.501184 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr9ld_68a3d167-dbc7-4155-aee3-e056b558ab8a/serve-healthcheck-canary/0.log" Oct 30 18:31:24.704129 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.704097 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-ffzcc_85a0105a-b96c-449d-9e91-bd8e457c0a0a/migrator/0.log" Oct 30 18:31:24.716690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.716669 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-687ccc8fd4-p2sn5"] Oct 30 18:31:24.719361 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:24.719325 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6926c1_1692_4111_85b6_319433b9bdfe.slice/crio-4b8bbe513d3ffb3dff7a9b4bc001f8a2a7bb4b50205a74af820e8570a7f081e1 WatchSource:0}: Error finding container 4b8bbe513d3ffb3dff7a9b4bc001f8a2a7bb4b50205a74af820e8570a7f081e1: Status 404 returned error can't find the container with id 4b8bbe513d3ffb3dff7a9b4bc001f8a2a7bb4b50205a74af820e8570a7f081e1 Oct 30 18:31:24.730811 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.730783 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xlm58"] Oct 30 18:31:24.732650 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:24.732628 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ed4e5d_002c_48ca_ad1e_199694eef36b.slice/crio-28967735f742755cbfe720ef1f44c50208f57002a99c5ccf13f352c7bd7e3f9b WatchSource:0}: Error finding container 28967735f742755cbfe720ef1f44c50208f57002a99c5ccf13f352c7bd7e3f9b: Status 404 returned error can't find the container with id 28967735f742755cbfe720ef1f44c50208f57002a99c5ccf13f352c7bd7e3f9b Oct 30 18:31:24.734649 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.734629 5070 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:31:24.759843 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.759786 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:24.759843 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.759818 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.759845 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.759883 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.759919 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.759943 5070 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.759960 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.759997 5070 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760006 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs podName:1adf30d9-26c7-46b0-b9bb-dfb96ba3711e nodeName:}" failed. No retries permitted until 2026-10-30 18:31:25.759985572 +0000 UTC m=+2.875119490 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs") pod "router-default-75fdfd5b99-vpjd7" (UID: "1adf30d9-26c7-46b0-b9bb-dfb96ba3711e") : secret "router-metrics-certs-default" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760067 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:25.760035099 +0000 UTC m=+2.875169011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.759996 5070 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.760111 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760125 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:25.760113226 +0000 UTC m=+2.875247135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.759943 5070 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760200 5070 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760229 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:25.76021975 +0000 UTC m=+2.875353671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:24.760388 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760244 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:25.760236588 +0000 UTC m=+2.875370494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:24.761214 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760417 5070 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:24.761214 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:24.760452 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:25.760444215 +0000 UTC m=+2.875578133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:24.761763 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.761742 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-service-ca-bundle\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:24.762749 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.762723 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:24.763538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.763511 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd54f845-4vwhq"] Oct 30 18:31:24.766172 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:24.766135 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865d20d3_b981_4d85_a56b_4951ae770312.slice/crio-71ddb7ad55ff1e13a2b69322ed328152d1969b78fc92640e57b2f790e39be7c0 WatchSource:0}: Error finding container 71ddb7ad55ff1e13a2b69322ed328152d1969b78fc92640e57b2f790e39be7c0: Status 404 returned error can't find the container with id 71ddb7ad55ff1e13a2b69322ed328152d1969b78fc92640e57b2f790e39be7c0 Oct 30 18:31:24.895598 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.895563 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-zvks5_729c533b-1000-4d81-ad6e-a06a15679e78/migrator/0.log" Oct 30 18:31:24.939928 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.939896 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:24.983571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:24.983524 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:25.103250 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.103165 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8_30e186ca-854c-417a-96ed-3a660db8b8f4/kube-storage-version-migrator-operator/0.log" Oct 30 18:31:25.103805 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.103785 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:31:25.103929 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.103906 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" event={"ID":"eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3","Type":"ContainerStarted","Data":"dad3ab7e424b144a1ae369f76e7d623f44e3136fefa24a7b2b93f758dc3b87c4"} Oct 30 18:31:25.104258 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.104234 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:25.105727 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.105692 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" event={"ID":"865d20d3-b981-4d85-a56b-4951ae770312","Type":"ContainerStarted","Data":"340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d"} Oct 30 18:31:25.105843 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.105738 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" event={"ID":"865d20d3-b981-4d85-a56b-4951ae770312","Type":"ContainerStarted","Data":"71ddb7ad55ff1e13a2b69322ed328152d1969b78fc92640e57b2f790e39be7c0"} Oct 30 18:31:25.107983 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.107793 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" event={"ID":"9e6926c1-1692-4111-85b6-319433b9bdfe","Type":"ContainerStarted","Data":"3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9"} Oct 30 18:31:25.107983 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.107839 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" event={"ID":"9e6926c1-1692-4111-85b6-319433b9bdfe","Type":"ContainerStarted","Data":"4b8bbe513d3ffb3dff7a9b4bc001f8a2a7bb4b50205a74af820e8570a7f081e1"} Oct 30 18:31:25.109133 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.109105 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlm58" event={"ID":"10ed4e5d-002c-48ca-ad1e-199694eef36b","Type":"ContainerStarted","Data":"28967735f742755cbfe720ef1f44c50208f57002a99c5ccf13f352c7bd7e3f9b"} Oct 30 18:31:25.110028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.110011 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-c262b" Oct 30 18:31:25.112390 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.112368 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:25.309134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.309101 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:25.506826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.506795 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:25.666591 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.666515 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:25.771042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.771018 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:25.771158 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.771053 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:25.771158 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.771097 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:25.771232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.771176 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:25.771232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.771203 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:25.771232 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771211 5070 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:25.771232 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771214 5070 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.771237 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771259 5070 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771224 5070 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771274 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.771255832 +0000 UTC m=+4.886389747 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771292 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.771282532 +0000 UTC m=+4.886416433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771307 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls podName:79d5b339-6665-4fbe-92db-77fd202119d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.77129821 +0000 UTC m=+4.886432108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls") pod "dns-default-lpq29" (UID: "79d5b339-6665-4fbe-92db-77fd202119d2") : secret "dns-default-metrics-tls" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771322 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.771313607 +0000 UTC m=+4.886447505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771335 5070 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:31:25.771364 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:25.771363 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls podName:56a08134-6094-4349-b252-ccf6d0aa8774 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:27.771356334 +0000 UTC m=+4.886490247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-2qkfc" (UID: "56a08134-6094-4349-b252-ccf6d0aa8774") : secret "samples-operator-tls" not found Oct 30 18:31:25.773708 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.773685 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1adf30d9-26c7-46b0-b9bb-dfb96ba3711e-metrics-certs\") pod \"router-default-75fdfd5b99-vpjd7\" (UID: \"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e\") " pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:25.844536 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.844508 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:25.861791 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:25.861762 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:26.026026 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.025953 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:26.114708 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.114648 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlm58" event={"ID":"10ed4e5d-002c-48ca-ad1e-199694eef36b","Type":"ContainerStarted","Data":"ee63f234faeb3caa47a085c00e0f11b5b11c2668f95a87fbee01e7400e1c73c5"} Oct 30 18:31:26.115338 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.114728 5070 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:26.115338 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.114793 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xlm58" event={"ID":"10ed4e5d-002c-48ca-ad1e-199694eef36b","Type":"ContainerStarted","Data":"4fcc4fab0cd5aa9b4255eb3d33c72603618fc4d50aa87142bc82f7e7690bc5cc"} Oct 30 18:31:26.149569 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.149544 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:26.193193 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.193167 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75fdfd5b99-vpjd7"] Oct 30 18:31:26.193679 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:26.193653 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1adf30d9_26c7_46b0_b9bb_dfb96ba3711e.slice/crio-4fbabeb38f96ce23e29b41be936b03c4989f33f602f2b52201967187c0bdc176 WatchSource:0}: Error finding container 4fbabeb38f96ce23e29b41be936b03c4989f33f602f2b52201967187c0bdc176: Status 404 returned error can't find the container with id 4fbabeb38f96ce23e29b41be936b03c4989f33f602f2b52201967187c0bdc176 Oct 30 18:31:26.349429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.349359 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:26.379724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.379691 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:26.407696 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.407667 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:26.626044 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.626012 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:26.936774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:26.936694 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:27.049767 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.049730 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:27.121385 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.121344 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" event={"ID":"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e","Type":"ContainerStarted","Data":"2a71097168f199230871571912e78a607e14aace64adaa70c1ba70b1f669500b"} Oct 30 18:31:27.121566 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.121400 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" event={"ID":"1adf30d9-26c7-46b0-b9bb-dfb96ba3711e","Type":"ContainerStarted","Data":"4fbabeb38f96ce23e29b41be936b03c4989f33f602f2b52201967187c0bdc176"} Oct 30 18:31:27.122071 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.122049 5070 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:27.609878 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.609844 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:27.764996 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.764966 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:27.790065 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.790037 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:27.790232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.790091 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:27.790232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.790120 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:27.790232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.790170 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:27.790232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.790203 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:27.790443 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:27.790244 5070 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:31:27.790443 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:27.790312 5070 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:31:27.790443 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:27.790316 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert podName:2fdf2c77-42d8-4686-a656-334557d55531 nodeName:}" failed. No retries permitted until 2026-10-30 18:31:31.790296665 +0000 UTC m=+8.905430577 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-glvc2" (UID: "2fdf2c77-42d8-4686-a656-334557d55531") : secret "networking-console-plugin-cert" not found Oct 30 18:31:27.790443 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:27.790336 5070 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:27.790443 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:27.790353 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls podName:cbf77bfc-9322-4635-9080-36631d7613df nodeName:}" failed. No retries permitted until 2026-10-30 18:31:31.790343504 +0000 UTC m=+8.905477399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-srvzm" (UID: "cbf77bfc-9322-4635-9080-36631d7613df") : secret "insights-runtime-extractor-tls" not found Oct 30 18:31:27.790443 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:31:27.790401 5070 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls podName:06297482-f53e-497f-8012-1c6a55b1462b nodeName:}" failed. No retries permitted until 2026-10-30 18:31:31.790381498 +0000 UTC m=+8.905515402 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n922x" (UID: "06297482-f53e-497f-8012-1c6a55b1462b") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:31:27.792473 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.792453 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79d5b339-6665-4fbe-92db-77fd202119d2-metrics-tls\") pod \"dns-default-lpq29\" (UID: \"79d5b339-6665-4fbe-92db-77fd202119d2\") " pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:27.792668 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.792648 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/56a08134-6094-4349-b252-ccf6d0aa8774-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-2qkfc\" (UID: \"56a08134-6094-4349-b252-ccf6d0aa8774\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:27.862369 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.862303 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:27.865019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.864998 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:27.925293 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.925267 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:27.962940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.962920 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" Oct 30 18:31:27.975628 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:27.975606 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:28.145161 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:28.145100 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:28.263480 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:28.263452 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc"] Oct 30 18:31:28.291188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:28.291139 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lpq29"] Oct 30 18:31:28.292529 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:28.292506 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d5b339_6665_4fbe_92db_77fd202119d2.slice/crio-740f4001daf59bfdae6888979725350f2e20c8973b4f76189d78572acb316aff WatchSource:0}: Error finding container 740f4001daf59bfdae6888979725350f2e20c8973b4f76189d78572acb316aff: Status 404 returned error can't find the container with id 740f4001daf59bfdae6888979725350f2e20c8973b4f76189d78572acb316aff Oct 30 18:31:28.361104 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:28.361060 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:28.529253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:28.529175 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:28.969055 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:28.968984 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:29.133111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.133072 5070 generic.go:356] "Generic (PLEG): container finished" podID="13093b2f-a782-4521-b80f-f4ee194af653" containerID="b4fc2b96770a2989ad974c312aafe702811fe06b2883a953f3b085a6af6560ce" exitCode=0 Oct 30 18:31:29.133281 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.133246 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" event={"ID":"13093b2f-a782-4521-b80f-f4ee194af653","Type":"ContainerDied","Data":"b4fc2b96770a2989ad974c312aafe702811fe06b2883a953f3b085a6af6560ce"} Oct 30 18:31:29.133654 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.133633 5070 scope.go:122] "RemoveContainer" containerID="b4fc2b96770a2989ad974c312aafe702811fe06b2883a953f3b085a6af6560ce" Oct 30 18:31:29.135509 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.135484 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" event={"ID":"56a08134-6094-4349-b252-ccf6d0aa8774","Type":"ContainerStarted","Data":"f573b46bc4effca6d903b5931fd406718aca906f7c2a0de43e94e93fd920db58"} Oct 30 18:31:29.138879 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.138854 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:29.139463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.139437 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lpq29" event={"ID":"79d5b339-6665-4fbe-92db-77fd202119d2","Type":"ContainerStarted","Data":"740f4001daf59bfdae6888979725350f2e20c8973b4f76189d78572acb316aff"} Oct 30 18:31:29.139573 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.139496 5070 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:29.397285 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.397246 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:29.734762 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.734727 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:29.900031 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:29.899265 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:30.149102 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.148376 5070 generic.go:356] "Generic (PLEG): container finished" podID="19c4a917-c2ba-46e3-8b7f-d528b1c26bc9" containerID="2abe84ba6e4f91b36288785de2ba877a454d2a7d6682c1d57dbda7cfbc546199" exitCode=0 Oct 30 18:31:30.149102 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.148453 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" event={"ID":"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9","Type":"ContainerDied","Data":"2abe84ba6e4f91b36288785de2ba877a454d2a7d6682c1d57dbda7cfbc546199"} Oct 30 18:31:30.149102 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.148768 5070 scope.go:122] "RemoveContainer" containerID="2abe84ba6e4f91b36288785de2ba877a454d2a7d6682c1d57dbda7cfbc546199" Oct 30 18:31:30.152331 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.152064 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zkkkh" event={"ID":"13093b2f-a782-4521-b80f-f4ee194af653","Type":"ContainerStarted","Data":"54542c4137832c98ac62a5da3d18cb1a6abcdbefa1cdc14190cee8b685ced0cd"} Oct 30 18:31:30.154606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.154579 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" event={"ID":"56a08134-6094-4349-b252-ccf6d0aa8774","Type":"ContainerStarted","Data":"6f80530ebd87f3f354e8c02de07218f88b59eccad03901f9e5333d14eaf3f29f"} Oct 30 18:31:30.155783 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.155762 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:30.156662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.156643 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lpq29" event={"ID":"79d5b339-6665-4fbe-92db-77fd202119d2","Type":"ContainerStarted","Data":"040b24b13ffbbe4865f49597273a5e2c9fc65fd23b78d9f0bc0a21d253262458"} Oct 30 18:31:30.157069 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.157052 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:30.388370 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.388332 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:30.582045 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.582012 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:30.801406 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:30.801329 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:31.161602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.161565 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-vtvc9" event={"ID":"19c4a917-c2ba-46e3-8b7f-d528b1c26bc9","Type":"ContainerStarted","Data":"ea26ecd2ae43849fe1358c7062f023ebefe699de421183652a84b299dad04596"} Oct 30 18:31:31.163295 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.163271 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-2qkfc" event={"ID":"56a08134-6094-4349-b252-ccf6d0aa8774","Type":"ContainerStarted","Data":"e4bb9c60a663c93724d94c39af8de91e48a782cd8fb5e403111f467a221b42eb"} Oct 30 18:31:31.164936 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.164914 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lpq29" event={"ID":"79d5b339-6665-4fbe-92db-77fd202119d2","Type":"ContainerStarted","Data":"374016b817964f309c1874dda5a5170bc800a8a08ab6d0e4fad4e2d990cdbfd7"} Oct 30 18:31:31.825606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.825559 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:31.825606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.825613 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:31.826224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.825635 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:31.828301 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.828274 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cbf77bfc-9322-4635-9080-36631d7613df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-srvzm\" (UID: \"cbf77bfc-9322-4635-9080-36631d7613df\") " pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:31.828662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.828639 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/06297482-f53e-497f-8012-1c6a55b1462b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n922x\" (UID: \"06297482-f53e-497f-8012-1c6a55b1462b\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:31.828781 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.828760 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fdf2c77-42d8-4686-a656-334557d55531-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-glvc2\" (UID: \"2fdf2c77-42d8-4686-a656-334557d55531\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:31.863733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.863700 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" Oct 30 18:31:31.867663 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.867477 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" Oct 30 18:31:31.876469 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.876440 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-srvzm" Oct 30 18:31:31.949527 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.949449 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:31.949695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.949667 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:31.949953 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.949812 5070 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:31:31.951256 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:31.951231 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-75fdfd5b99-vpjd7" Oct 30 18:31:32.168935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.168897 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:32.180132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.180106 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-glvc2"] Oct 30 18:31:32.181551 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:32.181527 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdf2c77_42d8_4686_a656_334557d55531.slice/crio-81348c7bcd698d07e040e0da3474b9c3073071e9268231a32c76ae2075c7f7b7 WatchSource:0}: Error finding container 81348c7bcd698d07e040e0da3474b9c3073071e9268231a32c76ae2075c7f7b7: Status 404 returned error can't find the container with id 81348c7bcd698d07e040e0da3474b9c3073071e9268231a32c76ae2075c7f7b7 Oct 30 18:31:32.190892 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.190856 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n922x"] Oct 30 18:31:32.192636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.192614 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-srvzm"] Oct 30 18:31:32.192724 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:32.192683 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06297482_f53e_497f_8012_1c6a55b1462b.slice/crio-2f9ddfa4ae4aedb3edbbda7f83862ae2b5fe94d70bad493f5472062fd89a55d0 WatchSource:0}: Error finding container 2f9ddfa4ae4aedb3edbbda7f83862ae2b5fe94d70bad493f5472062fd89a55d0: Status 404 returned error can't find the container with id 2f9ddfa4ae4aedb3edbbda7f83862ae2b5fe94d70bad493f5472062fd89a55d0 Oct 30 18:31:32.194370 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:32.194330 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf77bfc_9322_4635_9080_36631d7613df.slice/crio-0a0776d728eb6794eda85ab33316180047f7457027e06586a9b973bd3e3c82d0 WatchSource:0}: Error finding container 0a0776d728eb6794eda85ab33316180047f7457027e06586a9b973bd3e3c82d0: Status 404 returned error can't find the container with id 0a0776d728eb6794eda85ab33316180047f7457027e06586a9b973bd3e3c82d0 Oct 30 18:31:32.349426 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.349398 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:32.520515 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.520418 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:32.718041 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.717998 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:32.916164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:32.916115 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:33.150192 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.150132 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:33.178420 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.178384 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" event={"ID":"2fdf2c77-42d8-4686-a656-334557d55531","Type":"ContainerStarted","Data":"81348c7bcd698d07e040e0da3474b9c3073071e9268231a32c76ae2075c7f7b7"} Oct 30 18:31:33.179962 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.179932 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" event={"ID":"06297482-f53e-497f-8012-1c6a55b1462b","Type":"ContainerStarted","Data":"2f9ddfa4ae4aedb3edbbda7f83862ae2b5fe94d70bad493f5472062fd89a55d0"} Oct 30 18:31:33.185753 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.185725 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srvzm" event={"ID":"cbf77bfc-9322-4635-9080-36631d7613df","Type":"ContainerStarted","Data":"a4c7a353b6e938bb4df107d525f60cb232553474da7977a9fe464e309bbcf049"} Oct 30 18:31:33.185880 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.185783 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srvzm" event={"ID":"cbf77bfc-9322-4635-9080-36631d7613df","Type":"ContainerStarted","Data":"00a10a9bb1b3b093b90b13c82ea0a162dc7f0a2bf63ea1f692b951e4238e5294"} Oct 30 18:31:33.185880 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.185815 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srvzm" event={"ID":"cbf77bfc-9322-4635-9080-36631d7613df","Type":"ContainerStarted","Data":"0a0776d728eb6794eda85ab33316180047f7457027e06586a9b973bd3e3c82d0"} Oct 30 18:31:33.325389 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.325363 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:33.370694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:33.370666 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6tz8" Oct 30 18:31:34.193462 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:34.193421 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-glvc2" event={"ID":"2fdf2c77-42d8-4686-a656-334557d55531","Type":"ContainerStarted","Data":"f8b8a8f388c00c6d3b24e81061ba288ea6becf49a5683f9f4374d641df903106"} Oct 30 18:31:34.195749 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:34.195712 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n922x" event={"ID":"06297482-f53e-497f-8012-1c6a55b1462b","Type":"ContainerStarted","Data":"0f2388f2768c1eab5d83b2319d04fc2e4832e52b548db968eb1efa77ad702985"} Oct 30 18:31:34.475408 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:34.475375 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-plcdm" Oct 30 18:31:34.656456 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:34.656423 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:34.834520 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:34.834344 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:35.038304 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:35.038220 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:35.201777 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:35.201735 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-srvzm" event={"ID":"cbf77bfc-9322-4635-9080-36631d7613df","Type":"ContainerStarted","Data":"c25f75beeac446c158d10a48caa5e3cc5d88bfa42961c4c40f929385eb389c2c"} Oct 30 18:31:35.269625 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:35.269588 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:35.922224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:35.922182 5070 patch_prober.go:29] interesting pod/image-registry-687ccc8fd4-p2sn5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 18:31:35.922388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:35.922254 5070 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:31:38.818752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:38.818717 5070 patch_prober.go:29] interesting pod/image-registry-6dd54f845-4vwhq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 18:31:38.819138 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:38.818767 5070 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" podUID="865d20d3-b981-4d85-a56b-4951ae770312" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:31:39.903489 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:39.903457 5070 patch_prober.go:29] interesting pod/image-registry-6dd54f845-4vwhq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 18:31:39.903898 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:39.903508 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" podUID="865d20d3-b981-4d85-a56b-4951ae770312" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:31:40.986288 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:40.986246 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-6jh2k-wg5tb"] Oct 30 18:31:40.994709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:40.994684 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.011215 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.011185 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-xsmnj"] Oct 30 18:31:41.014007 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.013981 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.024072 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.024043 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f9444f6d9-fjllt"] Oct 30 18:31:41.028536 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.028509 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-6jh2k-wg5tb"] Oct 30 18:31:41.028663 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.028632 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.035173 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.035123 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-687ccc8fd4-p2sn5"] Oct 30 18:31:41.039774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.039750 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 18:31:41.043047 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.041430 5070 patch_prober.go:29] interesting pod/image-registry-687ccc8fd4-p2sn5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 18:31:41.043443 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.043217 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:31:41.043622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.043003 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 18:31:41.043721 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.042939 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-xsmnj"] Oct 30 18:31:41.048289 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.048268 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f9444f6d9-fjllt"] Oct 30 18:31:41.049296 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.049261 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s77r6\"" Oct 30 18:31:41.050274 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.050237 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 18:31:41.050499 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.050487 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-spdl5\"" Oct 30 18:31:41.058852 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.058134 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 18:31:41.058852 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.058293 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 18:31:41.059270 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.059239 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 18:31:41.100516 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.100483 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm92h\" (UniqueName: \"kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.100516 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.100522 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.100722 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.100617 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.100785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.100765 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.102743 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.102718 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl"] Oct 30 18:31:41.105075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.105056 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-566c94599c-nvz29"] Oct 30 18:31:41.105231 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.105214 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:41.107302 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.107281 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.135283 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.130220 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:41.145779 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.145737 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl"] Oct 30 18:31:41.148283 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.148258 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-566c94599c-nvz29"] Oct 30 18:31:41.153371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.153345 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 18:31:41.153694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.153668 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-56plm\"" Oct 30 18:31:41.153795 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.153700 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Oct 30 18:31:41.172892 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.172293 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lpq29" Oct 30 18:31:41.202736 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.202699 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zpth\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-kube-api-access-4zpth\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.202884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.202766 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.202884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.202810 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.202985 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.202882 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hh2s\" (UniqueName: \"kubernetes.io/projected/7a1e5faa-e3b4-4764-bd0d-6717de32fda3-kube-api-access-5hh2s\") pod \"downloads-6dd789c8b7-xsmnj\" (UID: \"7a1e5faa-e3b4-4764-bd0d-6717de32fda3\") " pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.202985 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.202938 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.203064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203007 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9x5d\" (UniqueName: \"kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.203179 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203140 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-registry-tls\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203231 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5550d6f-177d-40fe-8a26-7a369d3b5208-ca-trust-extracted\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203286 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203255 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5550d6f-177d-40fe-8a26-7a369d3b5208-installation-pull-secrets\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203319 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203288 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm92h\" (UniqueName: \"kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.203319 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203306 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.203380 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203327 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.203380 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203350 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-bound-sa-token\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203380 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203367 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a1e5faa-e3b4-4764-bd0d-6717de32fda3-tmp\") pod \"downloads-6dd789c8b7-xsmnj\" (UID: \"7a1e5faa-e3b4-4764-bd0d-6717de32fda3\") " pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.203476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203381 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.203476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203400 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2969e9ce-6c93-4660-a5ee-050390b1aec8-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pprwl\" (UID: \"2969e9ce-6c93-4660-a5ee-050390b1aec8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:41.203476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203418 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.203476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203436 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5550d6f-177d-40fe-8a26-7a369d3b5208-trusted-ca\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203455 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.203656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203478 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5550d6f-177d-40fe-8a26-7a369d3b5208-image-registry-private-configuration\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203495 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5550d6f-177d-40fe-8a26-7a369d3b5208-registry-certificates\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.203656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.203522 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.204589 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.204558 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.204721 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.204668 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.204897 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.204875 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.240472 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.240389 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm92h\" (UniqueName: \"kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h\") pod \"periodic-gathering-6jh2k-wg5tb\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.304049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304016 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304053 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304072 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hh2s\" (UniqueName: \"kubernetes.io/projected/7a1e5faa-e3b4-4764-bd0d-6717de32fda3-kube-api-access-5hh2s\") pod \"downloads-6dd789c8b7-xsmnj\" (UID: \"7a1e5faa-e3b4-4764-bd0d-6717de32fda3\") " pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.304440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304193 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9x5d\" (UniqueName: \"kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304256 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-registry-tls\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304310 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5550d6f-177d-40fe-8a26-7a369d3b5208-ca-trust-extracted\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304346 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5550d6f-177d-40fe-8a26-7a369d3b5208-installation-pull-secrets\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304410 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304466 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-bound-sa-token\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304493 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a1e5faa-e3b4-4764-bd0d-6717de32fda3-tmp\") pod \"downloads-6dd789c8b7-xsmnj\" (UID: \"7a1e5faa-e3b4-4764-bd0d-6717de32fda3\") " pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304529 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304560 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2969e9ce-6c93-4660-a5ee-050390b1aec8-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pprwl\" (UID: \"2969e9ce-6c93-4660-a5ee-050390b1aec8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304599 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5550d6f-177d-40fe-8a26-7a369d3b5208-trusted-ca\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304629 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304667 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5550d6f-177d-40fe-8a26-7a369d3b5208-image-registry-private-configuration\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304693 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5550d6f-177d-40fe-8a26-7a369d3b5208-registry-certificates\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.304997 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304741 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.304997 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.304774 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zpth\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-kube-api-access-4zpth\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.305279 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.305252 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5550d6f-177d-40fe-8a26-7a369d3b5208-ca-trust-extracted\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.307443 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.305397 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a1e5faa-e3b4-4764-bd0d-6717de32fda3-tmp\") pod \"downloads-6dd789c8b7-xsmnj\" (UID: \"7a1e5faa-e3b4-4764-bd0d-6717de32fda3\") " pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.307606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.305987 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.307715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.306273 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.307715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.307408 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5550d6f-177d-40fe-8a26-7a369d3b5208-trusted-ca\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.307838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.307506 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5550d6f-177d-40fe-8a26-7a369d3b5208-registry-certificates\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.307838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.307648 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.307838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.307793 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.308538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.308488 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-registry-tls\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.308623 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.308560 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.310372 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.310096 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5550d6f-177d-40fe-8a26-7a369d3b5208-image-registry-private-configuration\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.310372 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.310207 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.310372 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.310236 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2969e9ce-6c93-4660-a5ee-050390b1aec8-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pprwl\" (UID: \"2969e9ce-6c93-4660-a5ee-050390b1aec8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:41.310575 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.310434 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5550d6f-177d-40fe-8a26-7a369d3b5208-installation-pull-secrets\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.314920 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.314888 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:41.323358 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.323327 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:31:41.340118 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.340068 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9x5d\" (UniqueName: \"kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d\") pod \"console-5f9444f6d9-fjllt\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.341830 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.341777 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zpth\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-kube-api-access-4zpth\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.342416 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.342370 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5550d6f-177d-40fe-8a26-7a369d3b5208-bound-sa-token\") pod \"image-registry-566c94599c-nvz29\" (UID: \"a5550d6f-177d-40fe-8a26-7a369d3b5208\") " pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.361168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.350826 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:41.361168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.353999 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hh2s\" (UniqueName: \"kubernetes.io/projected/7a1e5faa-e3b4-4764-bd0d-6717de32fda3-kube-api-access-5hh2s\") pod \"downloads-6dd789c8b7-xsmnj\" (UID: \"7a1e5faa-e3b4-4764-bd0d-6717de32fda3\") " pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.423378 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.423341 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:41.434570 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.433630 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:41.497069 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.496863 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:41.634990 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.634952 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:41.666760 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.666728 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-6jh2k-wg5tb"] Oct 30 18:31:41.675454 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:41.675413 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a201b9_118c_43aa_8be1_1329c22d9130.slice/crio-00869ff1329ded85d7c920cfef789b1406c10eac025df27ddda9b1b96b3316e4 WatchSource:0}: Error finding container 00869ff1329ded85d7c920cfef789b1406c10eac025df27ddda9b1b96b3316e4: Status 404 returned error can't find the container with id 00869ff1329ded85d7c920cfef789b1406c10eac025df27ddda9b1b96b3316e4 Oct 30 18:31:41.718059 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.716418 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f9444f6d9-fjllt"] Oct 30 18:31:41.743927 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:41.743895 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee8c4e7_fcbf_48d6_aa68_6294521aa485.slice/crio-3e2d2b1ade72d68a75268cb5a7e7a25bde001ce1e288ece942f9d5b079c9ff76 WatchSource:0}: Error finding container 3e2d2b1ade72d68a75268cb5a7e7a25bde001ce1e288ece942f9d5b079c9ff76: Status 404 returned error can't find the container with id 3e2d2b1ade72d68a75268cb5a7e7a25bde001ce1e288ece942f9d5b079c9ff76 Oct 30 18:31:41.813615 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.813528 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl"] Oct 30 18:31:41.857884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.857777 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-566c94599c-nvz29"] Oct 30 18:31:41.860327 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:41.860295 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5550d6f_177d_40fe_8a26_7a369d3b5208.slice/crio-16bc32c644541e9786c4a9834f47f3bf317655d142fa3d20c4797db6672e3175 WatchSource:0}: Error finding container 16bc32c644541e9786c4a9834f47f3bf317655d142fa3d20c4797db6672e3175: Status 404 returned error can't find the container with id 16bc32c644541e9786c4a9834f47f3bf317655d142fa3d20c4797db6672e3175 Oct 30 18:31:41.963669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:41.963638 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-xsmnj"] Oct 30 18:31:41.969715 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:41.969689 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a1e5faa_e3b4_4764_bd0d_6717de32fda3.slice/crio-d78e18266d94b72eb03c4c7f35680484bd246424cb681df03eff98f26c7917ec WatchSource:0}: Error finding container d78e18266d94b72eb03c4c7f35680484bd246424cb681df03eff98f26c7917ec: Status 404 returned error can't find the container with id d78e18266d94b72eb03c4c7f35680484bd246424cb681df03eff98f26c7917ec Oct 30 18:31:42.253749 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.253708 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-xsmnj" event={"ID":"7a1e5faa-e3b4-4764-bd0d-6717de32fda3","Type":"ContainerStarted","Data":"d78e18266d94b72eb03c4c7f35680484bd246424cb681df03eff98f26c7917ec"} Oct 30 18:31:42.255134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.255102 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" event={"ID":"2969e9ce-6c93-4660-a5ee-050390b1aec8","Type":"ContainerStarted","Data":"28693156c4302bdd8d3b7bd6c6c6209c785eaea5ec7f102c83eb68839a8a6dfe"} Oct 30 18:31:42.258045 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.256553 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9444f6d9-fjllt" event={"ID":"3ee8c4e7-fcbf-48d6-aa68-6294521aa485","Type":"ContainerStarted","Data":"3e2d2b1ade72d68a75268cb5a7e7a25bde001ce1e288ece942f9d5b079c9ff76"} Oct 30 18:31:42.258604 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.258581 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-566c94599c-nvz29" event={"ID":"a5550d6f-177d-40fe-8a26-7a369d3b5208","Type":"ContainerStarted","Data":"a886ad2ff35da64ea64f18a2d8a7ba9b716e01b5c495ebe06fd29c4c2c89c476"} Oct 30 18:31:42.258730 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.258619 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-566c94599c-nvz29" event={"ID":"a5550d6f-177d-40fe-8a26-7a369d3b5208","Type":"ContainerStarted","Data":"16bc32c644541e9786c4a9834f47f3bf317655d142fa3d20c4797db6672e3175"} Oct 30 18:31:42.258730 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.258670 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:31:42.261491 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.260501 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" event={"ID":"89a201b9-118c-43aa-8be1-1329c22d9130","Type":"ContainerStarted","Data":"b7f88f1f21b509bf7d139a1a0c164e6ccbd975011a86c42cd8b5eb6a902d342b"} Oct 30 18:31:42.261491 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.260541 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" event={"ID":"89a201b9-118c-43aa-8be1-1329c22d9130","Type":"ContainerStarted","Data":"00869ff1329ded85d7c920cfef789b1406c10eac025df27ddda9b1b96b3316e4"} Oct 30 18:31:42.305060 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.305007 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-566c94599c-nvz29" podStartSLOduration=1.304992989 podStartE2EDuration="1.304992989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:42.304915044 +0000 UTC m=+19.420048962" watchObservedRunningTime="2026-10-30 18:31:42.304992989 +0000 UTC m=+19.420126903" Oct 30 18:31:42.352837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.352768 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" podStartSLOduration=2.352748529 podStartE2EDuration="2.352748529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:42.351589919 +0000 UTC m=+19.466723837" watchObservedRunningTime="2026-10-30 18:31:42.352748529 +0000 UTC m=+19.467882448" Oct 30 18:31:42.797210 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.797169 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:42.976433 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:42.976346 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:43.271209 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:43.271083 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" event={"ID":"2969e9ce-6c93-4660-a5ee-050390b1aec8","Type":"ContainerStarted","Data":"37918f697810d2e511a5969b113a3adae8839793121cd96e6ac3dced00b97207"} Oct 30 18:31:43.272163 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:43.272115 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:43.277527 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:43.277490 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" Oct 30 18:31:43.314636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:43.314593 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pprwl" podStartSLOduration=1.0186659 podStartE2EDuration="2.314581524s" totalImagesPullingTime="1.295915624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:43.313254091 +0000 UTC m=+20.428388011" watchObservedRunningTime="2026-10-30 18:31:43.314581524 +0000 UTC m=+20.429715435" Oct 30 18:31:43.956661 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:43.956623 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:44.108538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:44.108498 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:44.312906 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:44.312833 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:44.512704 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:44.512665 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:45.175663 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:45.175631 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lpq29_79d5b339-6665-4fbe-92db-77fd202119d2/dns/0.log" Oct 30 18:31:45.283059 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:45.283020 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9444f6d9-fjllt" event={"ID":"3ee8c4e7-fcbf-48d6-aa68-6294521aa485","Type":"ContainerStarted","Data":"9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4"} Oct 30 18:31:45.327417 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:45.327323 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5f9444f6d9-fjllt" podStartSLOduration=2.004455478 podStartE2EDuration="5.327309625s" totalImagesPullingTime="3.322854147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:45.326664348 +0000 UTC m=+22.441798284" watchObservedRunningTime="2026-10-30 18:31:45.327309625 +0000 UTC m=+22.442443539" Oct 30 18:31:45.365915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:45.365887 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lpq29_79d5b339-6665-4fbe-92db-77fd202119d2/kube-rbac-proxy/0.log" Oct 30 18:31:46.362906 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:46.362881 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sm8gj_463c6228-727e-4d4f-9b80-1db615657b4f/dns-node-resolver/0.log" Oct 30 18:31:46.562886 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:46.562852 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-566c94599c-nvz29_a5550d6f-177d-40fe-8a26-7a369d3b5208/registry/0.log" Oct 30 18:31:46.762868 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:46.762834 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-687ccc8fd4-p2sn5_9e6926c1-1692-4111-85b6-319433b9bdfe/registry/0.log" Oct 30 18:31:46.938169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:46.938112 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:46.963575 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:46.963546 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6dd54f845-4vwhq_865d20d3-b981-4d85-a56b-4951ae770312/registry/0.log" Oct 30 18:31:47.162264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:47.162236 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9nr4w_11d10c3e-223f-41ef-ac3f-cd428bf43ebd/node-ca/0.log" Oct 30 18:31:47.767261 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:47.767215 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:31:47.814800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:47.814764 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:48.016562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:48.016522 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:48.162611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:48.162577 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr9ld_68a3d167-dbc7-4155-aee3-e056b558ab8a/serve-healthcheck-canary/0.log" Oct 30 18:31:48.625538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:48.625501 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:48.933068 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:48.932990 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:48.978485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:48.978456 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hg6zs"] Oct 30 18:31:48.984286 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:48.984263 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.048448 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.048415 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:31:49.048863 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.048785 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:31:49.048863 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.048823 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:31:49.048863 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.048824 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twcvb\"" Oct 30 18:31:49.060808 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.060787 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:31:49.095934 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.095903 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-textfile\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.095946 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr4fv\" (UniqueName: \"kubernetes.io/projected/7990fc23-358e-4104-b308-12adc1dcbc9a-kube-api-access-nr4fv\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096062 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7990fc23-358e-4104-b308-12adc1dcbc9a-metrics-client-ca\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096102 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-root\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096126 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-wtmp\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096208 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-tls\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096242 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-sys\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096394 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096317 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.096394 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.096359 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.196935 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7990fc23-358e-4104-b308-12adc1dcbc9a-metrics-client-ca\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.196976 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-root\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.196992 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-wtmp\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197287 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197073 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-root\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197287 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197209 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-tls\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197287 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197218 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-wtmp\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197343 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-sys\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197391 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197429 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7990fc23-358e-4104-b308-12adc1dcbc9a-sys\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197436 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197536 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-textfile\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197564 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr4fv\" (UniqueName: \"kubernetes.io/projected/7990fc23-358e-4104-b308-12adc1dcbc9a-kube-api-access-nr4fv\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197776 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197748 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7990fc23-358e-4104-b308-12adc1dcbc9a-metrics-client-ca\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.197903 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.197887 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-textfile\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.198097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.198072 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.199839 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.199817 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.200314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.200294 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7990fc23-358e-4104-b308-12adc1dcbc9a-node-exporter-tls\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.221956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.221926 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr4fv\" (UniqueName: \"kubernetes.io/projected/7990fc23-358e-4104-b308-12adc1dcbc9a-kube-api-access-nr4fv\") pod \"node-exporter-hg6zs\" (UID: \"7990fc23-358e-4104-b308-12adc1dcbc9a\") " pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.242249 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.242217 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:49.303678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.303649 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hg6zs" Oct 30 18:31:49.326277 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:49.326252 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7990fc23_358e_4104_b308_12adc1dcbc9a.slice/crio-3652eeab85571ddb6fb02a5e6ca90ce717a51dcd9a879e518db04517b942d9fe WatchSource:0}: Error finding container 3652eeab85571ddb6fb02a5e6ca90ce717a51dcd9a879e518db04517b942d9fe: Status 404 returned error can't find the container with id 3652eeab85571ddb6fb02a5e6ca90ce717a51dcd9a879e518db04517b942d9fe Oct 30 18:31:49.409034 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.408998 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:49.646843 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.646806 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:49.906747 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:49.905995 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:31:50.016180 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:50.014555 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:50.297299 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:50.297223 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:50.310190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:50.310127 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg6zs" event={"ID":"7990fc23-358e-4104-b308-12adc1dcbc9a","Type":"ContainerStarted","Data":"3652eeab85571ddb6fb02a5e6ca90ce717a51dcd9a879e518db04517b942d9fe"} Oct 30 18:31:50.478290 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:50.478254 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:51.040342 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:51.040313 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:31:51.320882 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:51.320808 5070 generic.go:356] "Generic (PLEG): container finished" podID="7990fc23-358e-4104-b308-12adc1dcbc9a" containerID="aacd5611662aafd966ad1a65fd922a79843b4ee508396cd969459f6cb4a4d5b3" exitCode=0 Oct 30 18:31:51.321041 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:51.320882 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg6zs" event={"ID":"7990fc23-358e-4104-b308-12adc1dcbc9a","Type":"ContainerDied","Data":"aacd5611662aafd966ad1a65fd922a79843b4ee508396cd969459f6cb4a4d5b3"} Oct 30 18:31:51.354127 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:51.354098 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:51.354250 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:51.354232 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:51.360188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:51.360140 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:52.203967 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:52.203919 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:52.330798 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:52.330753 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg6zs" event={"ID":"7990fc23-358e-4104-b308-12adc1dcbc9a","Type":"ContainerStarted","Data":"0c583afce268c120999d0636623eb3ec1e62e17ba102b51daa444a75052d983d"} Oct 30 18:31:52.330958 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:52.330821 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hg6zs" event={"ID":"7990fc23-358e-4104-b308-12adc1dcbc9a","Type":"ContainerStarted","Data":"c24a5483711cd52020809266234723282d2384c9a4f110e37843be203ec693e9"} Oct 30 18:31:52.336428 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:52.336390 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:31:52.377302 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:52.377268 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:52.440609 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:52.440553 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hg6zs" podStartSLOduration=3.170800009 podStartE2EDuration="4.440533828s" totalImagesPullingTime="1.18788477s" totalInitContainerRuntime="81.849049ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:52.396046483 +0000 UTC m=+29.511180399" watchObservedRunningTime="2026-10-30 18:31:52.440533828 +0000 UTC m=+29.555667742" Oct 30 18:31:53.700902 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:53.700865 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:53.841924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:53.841884 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:54.029396 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:54.029319 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:55.977741 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:55.977706 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:56.411357 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.411327 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gwzzt" Oct 30 18:31:56.443787 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.443757 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-997f887b4-xcw6g"] Oct 30 18:31:56.482980 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.482923 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-997f887b4-xcw6g"] Oct 30 18:31:56.483162 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.483024 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.524748 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.524703 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 18:31:56.566345 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566314 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566513 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566373 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566513 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566488 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566526 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566548 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566687 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566823 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566763 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.566823 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.566814 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667362 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667440 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667414 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667454 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667477 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667518 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667538 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667566 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.667651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.667589 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.668810 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.668257 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.668810 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.668460 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.668810 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.668637 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.668810 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.668764 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.669035 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.668855 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.670264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.670243 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.671936 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.671914 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.693608 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.693586 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm\") pod \"console-997f887b4-xcw6g\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:56.813290 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:56.813248 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:31:57.140025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:57.140001 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-997f887b4-xcw6g"] Oct 30 18:31:57.142795 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:31:57.142768 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843291f5_2d22_4ba5_b99b_b499cca25268.slice/crio-fdc837388ec169a0f7f9cbf78243966d5cdfb28cfd746920300147fe238d02fa WatchSource:0}: Error finding container fdc837388ec169a0f7f9cbf78243966d5cdfb28cfd746920300147fe238d02fa: Status 404 returned error can't find the container with id fdc837388ec169a0f7f9cbf78243966d5cdfb28cfd746920300147fe238d02fa Oct 30 18:31:57.356790 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:57.356751 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997f887b4-xcw6g" event={"ID":"843291f5-2d22-4ba5-b99b-b499cca25268","Type":"ContainerStarted","Data":"3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c"} Oct 30 18:31:57.357045 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:57.356814 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997f887b4-xcw6g" event={"ID":"843291f5-2d22-4ba5-b99b-b499cca25268","Type":"ContainerStarted","Data":"fdc837388ec169a0f7f9cbf78243966d5cdfb28cfd746920300147fe238d02fa"} Oct 30 18:31:57.401005 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:57.399742 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-997f887b4-xcw6g" podStartSLOduration=1.399726392 podStartE2EDuration="1.399726392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:57.399033385 +0000 UTC m=+34.514167328" watchObservedRunningTime="2026-10-30 18:31:57.399726392 +0000 UTC m=+34.514860308" Oct 30 18:31:59.057039 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.057006 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:59.320816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.320750 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:31:59.367936 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.367902 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-xsmnj" event={"ID":"7a1e5faa-e3b4-4764-bd0d-6717de32fda3","Type":"ContainerStarted","Data":"d022ad39101b568b528f9f1377a86376967917a12314cb3837a6aebf821e7719"} Oct 30 18:31:59.368130 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.368111 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:31:59.369593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.369574 5070 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-xsmnj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.29:8080/\": dial tcp 10.132.0.29:8080: connect: connection refused" start-of-body= Oct 30 18:31:59.369669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.369645 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-xsmnj" podUID="7a1e5faa-e3b4-4764-bd0d-6717de32fda3" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.29:8080/\": dial tcp 10.132.0.29:8080: connect: connection refused" Oct 30 18:31:59.428636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:31:59.428583 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-xsmnj" podStartSLOduration=2.239403754 podStartE2EDuration="19.428571565s" totalImagesPullingTime="17.189167811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:31:59.427601868 +0000 UTC m=+36.542735803" watchObservedRunningTime="2026-10-30 18:31:59.428571565 +0000 UTC m=+36.543705475" Oct 30 18:32:00.388798 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:00.388764 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-xsmnj" Oct 30 18:32:01.361050 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:01.361011 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:01.541388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:01.541354 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:01.870720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:01.870684 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:02.051108 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:02.051071 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-997f887b4-xcw6g"] Oct 30 18:32:02.162260 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:02.162094 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:03.283183 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:03.282872 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-566c94599c-nvz29" Oct 30 18:32:03.388020 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:03.387984 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dd54f845-4vwhq"] Oct 30 18:32:03.557398 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:03.557283 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:03.839474 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:03.839386 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:32:03.862531 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:03.862497 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:04.104228 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:04.104134 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:04.319186 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:04.318635 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:05.183606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:05.183361 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:05.488033 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:05.487953 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:06.080208 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.079798 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerName="registry" containerID="cri-o://3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9" gracePeriod=30 Oct 30 18:32:06.411781 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.411744 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:32:06.426961 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.426925 5070 generic.go:356] "Generic (PLEG): container finished" podID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerID="3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9" exitCode=0 Oct 30 18:32:06.427124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.426972 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" event={"ID":"9e6926c1-1692-4111-85b6-319433b9bdfe","Type":"ContainerDied","Data":"3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9"} Oct 30 18:32:06.427124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.426980 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" Oct 30 18:32:06.427124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.427020 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-687ccc8fd4-p2sn5" event={"ID":"9e6926c1-1692-4111-85b6-319433b9bdfe","Type":"ContainerDied","Data":"4b8bbe513d3ffb3dff7a9b4bc001f8a2a7bb4b50205a74af820e8570a7f081e1"} Oct 30 18:32:06.427124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.427048 5070 scope.go:122] "RemoveContainer" containerID="3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9" Oct 30 18:32:06.440274 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.440244 5070 scope.go:122] "RemoveContainer" containerID="3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9" Oct 30 18:32:06.440845 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:06.440815 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9\": container with ID starting with 3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9 not found: ID does not exist" containerID="3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9" Oct 30 18:32:06.440944 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.440860 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9"} err="failed to get container status \"3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9\": rpc error: code = NotFound desc = could not find container \"3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9\": container with ID starting with 3a63416a0d5aaf35d453aa663cdb632a38a3c0a7f08f8d62244b7e6a9a1e76d9 not found: ID does not exist" Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475453 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475518 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475558 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475590 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475659 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475726 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475758 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.475791 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") pod \"9e6926c1-1692-4111-85b6-319433b9bdfe\" (UID: \"9e6926c1-1692-4111-85b6-319433b9bdfe\") " Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.479646 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.482275 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.482783 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:06.488485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.484269 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:06.493566 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.493511 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:06.493566 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.493534 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:06.494331 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.494297 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:06.495635 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.495613 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm" pod "9e6926c1-1692-4111-85b6-319433b9bdfe" (UID: "9e6926c1-1692-4111-85b6-319433b9bdfe"). InnerVolumeSpecName "kube-api-access-8m8gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:06.576518 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576477 5070 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-tls\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576518 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576511 5070 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-trusted-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576518 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576526 5070 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-bound-sa-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576537 5070 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e6926c1-1692-4111-85b6-319433b9bdfe-ca-trust-extracted\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576548 5070 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-installation-pull-secrets\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576559 5070 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e6926c1-1692-4111-85b6-319433b9bdfe-image-registry-private-configuration\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576575 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m8gm\" (UniqueName: \"kubernetes.io/projected/9e6926c1-1692-4111-85b6-319433b9bdfe-kube-api-access-8m8gm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.576819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.576590 5070 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e6926c1-1692-4111-85b6-319433b9bdfe-registry-certificates\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:06.781224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.781182 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-687ccc8fd4-p2sn5"] Oct 30 18:32:06.785359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.785329 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-687ccc8fd4-p2sn5"] Oct 30 18:32:06.813809 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:06.813778 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:32:07.032379 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:07.032291 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" path="/var/lib/kubelet/pods/9e6926c1-1692-4111-85b6-319433b9bdfe/volumes" Oct 30 18:32:10.017575 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:10.017544 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f9444f6d9-fjllt"] Oct 30 18:32:11.668013 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:11.667973 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:11.931261 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:11.931167 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:13.116163 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.116104 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:13.338224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.338180 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:13.408001 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.407974 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-v955r-lpqmq"] Oct 30 18:32:13.408189 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.408167 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-v955r-lpqmq" podUID="faa532be-e39c-4a3d-bf81-da68387b9703" containerName="insights-gathering" containerID="cri-o://0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1" gracePeriod=30 Oct 30 18:32:13.753018 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.752988 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:32:13.844621 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.844581 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") pod \"faa532be-e39c-4a3d-bf81-da68387b9703\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " Oct 30 18:32:13.844799 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.844643 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") pod \"faa532be-e39c-4a3d-bf81-da68387b9703\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " Oct 30 18:32:13.844799 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.844680 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") pod \"faa532be-e39c-4a3d-bf81-da68387b9703\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " Oct 30 18:32:13.844799 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.844725 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") pod \"faa532be-e39c-4a3d-bf81-da68387b9703\" (UID: \"faa532be-e39c-4a3d-bf81-da68387b9703\") " Oct 30 18:32:13.845004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.844976 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path" pod "faa532be-e39c-4a3d-bf81-da68387b9703" (UID: "faa532be-e39c-4a3d-bf81-da68387b9703"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:13.845077 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.845053 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle" pod "faa532be-e39c-4a3d-bf81-da68387b9703" (UID: "faa532be-e39c-4a3d-bf81-da68387b9703"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:13.845136 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.845063 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle" pod "faa532be-e39c-4a3d-bf81-da68387b9703" (UID: "faa532be-e39c-4a3d-bf81-da68387b9703"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:13.847341 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.847316 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7" pod "faa532be-e39c-4a3d-bf81-da68387b9703" (UID: "faa532be-e39c-4a3d-bf81-da68387b9703"). InnerVolumeSpecName "kube-api-access-755x7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:13.945773 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.945733 5070 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-service-ca-bundle\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:13.945773 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.945769 5070 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/faa532be-e39c-4a3d-bf81-da68387b9703-archives-path\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:13.945773 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.945779 5070 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faa532be-e39c-4a3d-bf81-da68387b9703-trusted-ca-bundle\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:13.946009 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:13.945788 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-755x7\" (UniqueName: \"kubernetes.io/projected/faa532be-e39c-4a3d-bf81-da68387b9703-kube-api-access-755x7\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:14.019752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.019668 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:14.208683 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.208652 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:14.465458 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.465428 5070 generic.go:356] "Generic (PLEG): container finished" podID="faa532be-e39c-4a3d-bf81-da68387b9703" containerID="0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1" exitCode=2 Oct 30 18:32:14.465629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.465467 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-v955r-lpqmq" Oct 30 18:32:14.465629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.465474 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v955r-lpqmq" event={"ID":"faa532be-e39c-4a3d-bf81-da68387b9703","Type":"ContainerDied","Data":"0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1"} Oct 30 18:32:14.465629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.465517 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-v955r-lpqmq" event={"ID":"faa532be-e39c-4a3d-bf81-da68387b9703","Type":"ContainerDied","Data":"9588ef34e32c3ec853ff3ac72f69bab7903190a025e3612ef5cd4cc4a96ce0f5"} Oct 30 18:32:14.465629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.465538 5070 scope.go:122] "RemoveContainer" containerID="0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1" Oct 30 18:32:14.474666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.474647 5070 scope.go:122] "RemoveContainer" containerID="0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1" Oct 30 18:32:14.475042 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:14.475019 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1\": container with ID starting with 0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1 not found: ID does not exist" containerID="0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1" Oct 30 18:32:14.475133 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.475051 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1"} err="failed to get container status \"0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1\": rpc error: code = NotFound desc = could not find container \"0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1\": container with ID starting with 0a03b7d46bcf522d2ef354ec88be0dba6c0c8c63692d15129a6d9f51202656d1 not found: ID does not exist" Oct 30 18:32:14.520028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.519997 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-v955r-lpqmq"] Oct 30 18:32:14.530913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:14.530887 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-v955r-lpqmq"] Oct 30 18:32:15.030689 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:15.030658 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="faa532be-e39c-4a3d-bf81-da68387b9703" path="/var/lib/kubelet/pods/faa532be-e39c-4a3d-bf81-da68387b9703/volumes" Oct 30 18:32:21.689734 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:21.689685 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:21.849576 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:21.849505 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:27.074774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.074735 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-997f887b4-xcw6g" podUID="843291f5-2d22-4ba5-b99b-b499cca25268" containerName="console" containerID="cri-o://3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c" gracePeriod=15 Oct 30 18:32:27.336447 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.336383 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:32:27.352098 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352070 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352114 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352175 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352197 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352223 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp\" (UniqueName: \"kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352453 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352260 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352453 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352297 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352453 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352369 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca\") pod \"843291f5-2d22-4ba5-b99b-b499cca25268\" (UID: \"843291f5-2d22-4ba5-b99b-b499cca25268\") " Oct 30 18:32:27.352615 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352516 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:27.352615 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352595 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:27.352745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352639 5070 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/843291f5-2d22-4ba5-b99b-b499cca25268-tmp\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.352745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352675 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:27.352962 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.352937 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:27.353049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.353000 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:27.354445 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.354417 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "kube-api-access-6h5qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:27.354529 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.354451 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:27.354529 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.354491 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert" pod "843291f5-2d22-4ba5-b99b-b499cca25268" (UID: "843291f5-2d22-4ba5-b99b-b499cca25268"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:27.453279 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453246 5070 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-service-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.453279 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453273 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/843291f5-2d22-4ba5-b99b-b499cca25268-kube-api-access-6h5qm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.453279 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453284 5070 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-console-config\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.453500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453293 5070 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-oauth-serving-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.453500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453302 5070 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-serving-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.453500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453312 5070 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843291f5-2d22-4ba5-b99b-b499cca25268-trusted-ca-bundle\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.453500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.453322 5070 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/843291f5-2d22-4ba5-b99b-b499cca25268-console-oauth-config\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:27.521095 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.521064 5070 generic.go:356] "Generic (PLEG): container finished" podID="843291f5-2d22-4ba5-b99b-b499cca25268" containerID="3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c" exitCode=0 Oct 30 18:32:27.521264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.521099 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997f887b4-xcw6g" event={"ID":"843291f5-2d22-4ba5-b99b-b499cca25268","Type":"ContainerDied","Data":"3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c"} Oct 30 18:32:27.521264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.521113 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-997f887b4-xcw6g" Oct 30 18:32:27.521264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.521137 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-997f887b4-xcw6g" event={"ID":"843291f5-2d22-4ba5-b99b-b499cca25268","Type":"ContainerDied","Data":"fdc837388ec169a0f7f9cbf78243966d5cdfb28cfd746920300147fe238d02fa"} Oct 30 18:32:27.521264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.521175 5070 scope.go:122] "RemoveContainer" containerID="3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c" Oct 30 18:32:27.533611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.533588 5070 scope.go:122] "RemoveContainer" containerID="3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c" Oct 30 18:32:27.533862 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:27.533843 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c\": container with ID starting with 3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c not found: ID does not exist" containerID="3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c" Oct 30 18:32:27.533967 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.533877 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c"} err="failed to get container status \"3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c\": rpc error: code = NotFound desc = could not find container \"3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c\": container with ID starting with 3a22fd1930aa774a69a8ca10e824d660b3bc144c551f3e6054ef66e9c684fc6c not found: ID does not exist" Oct 30 18:32:27.573477 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.573447 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-997f887b4-xcw6g"] Oct 30 18:32:27.577740 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:27.577720 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-997f887b4-xcw6g"] Oct 30 18:32:28.435063 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.435018 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" podUID="865d20d3-b981-4d85-a56b-4951ae770312" containerName="registry" containerID="cri-o://340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d" gracePeriod=30 Oct 30 18:32:28.703234 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.703171 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:32:28.763612 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763578 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763765 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763645 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763765 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763678 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763765 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763722 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763774 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763809 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763856 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.763940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.763898 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") pod \"865d20d3-b981-4d85-a56b-4951ae770312\" (UID: \"865d20d3-b981-4d85-a56b-4951ae770312\") " Oct 30 18:32:28.764169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.764091 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:28.764407 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.764376 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:28.766092 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.766067 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:28.766248 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.766064 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:28.766597 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.766573 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:28.766677 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.766599 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:28.766677 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.766655 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "kube-api-access-hnlkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:28.771090 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.770975 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted" pod "865d20d3-b981-4d85-a56b-4951ae770312" (UID: "865d20d3-b981-4d85-a56b-4951ae770312"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:28.864813 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864783 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnlkc\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-kube-api-access-hnlkc\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.864813 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864812 5070 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-image-registry-private-configuration\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.865008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864822 5070 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/865d20d3-b981-4d85-a56b-4951ae770312-ca-trust-extracted\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.865008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864832 5070 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-registry-certificates\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.865008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864842 5070 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/865d20d3-b981-4d85-a56b-4951ae770312-trusted-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.865008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864851 5070 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-bound-sa-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.865008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864859 5070 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/865d20d3-b981-4d85-a56b-4951ae770312-installation-pull-secrets\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:28.865008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:28.864868 5070 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/865d20d3-b981-4d85-a56b-4951ae770312-registry-tls\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:29.030813 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.030741 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="843291f5-2d22-4ba5-b99b-b499cca25268" path="/var/lib/kubelet/pods/843291f5-2d22-4ba5-b99b-b499cca25268/volumes" Oct 30 18:32:29.530873 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.530841 5070 generic.go:356] "Generic (PLEG): container finished" podID="865d20d3-b981-4d85-a56b-4951ae770312" containerID="340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d" exitCode=0 Oct 30 18:32:29.531347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.530886 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" Oct 30 18:32:29.531347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.530917 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" event={"ID":"865d20d3-b981-4d85-a56b-4951ae770312","Type":"ContainerDied","Data":"340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d"} Oct 30 18:32:29.531347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.530956 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd54f845-4vwhq" event={"ID":"865d20d3-b981-4d85-a56b-4951ae770312","Type":"ContainerDied","Data":"71ddb7ad55ff1e13a2b69322ed328152d1969b78fc92640e57b2f790e39be7c0"} Oct 30 18:32:29.531347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.530976 5070 scope.go:122] "RemoveContainer" containerID="340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d" Oct 30 18:32:29.539092 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.539074 5070 scope.go:122] "RemoveContainer" containerID="340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d" Oct 30 18:32:29.539386 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:29.539367 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d\": container with ID starting with 340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d not found: ID does not exist" containerID="340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d" Oct 30 18:32:29.539431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.539400 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d"} err="failed to get container status \"340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d\": rpc error: code = NotFound desc = could not find container \"340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d\": container with ID starting with 340241d199763a9f6170751b97f7bb88b0c90e4016c972471a45dfee63d0e43d not found: ID does not exist" Oct 30 18:32:29.598053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.598027 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dd54f845-4vwhq"] Oct 30 18:32:29.610651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:29.610626 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dd54f845-4vwhq"] Oct 30 18:32:31.029535 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:31.029507 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="865d20d3-b981-4d85-a56b-4951ae770312" path="/var/lib/kubelet/pods/865d20d3-b981-4d85-a56b-4951ae770312/volumes" Oct 30 18:32:33.831525 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:33.831476 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:32:34.823376 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:34.823327 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:34.981838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:34.981806 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:35.043702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.043670 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5f9444f6d9-fjllt" podUID="3ee8c4e7-fcbf-48d6-aa68-6294521aa485" containerName="console" containerID="cri-o://9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4" gracePeriod=15 Oct 30 18:32:35.346245 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.346187 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:32:35.374424 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.374390 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:35.418826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418802 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d\" (UniqueName: \"kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.418977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418846 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.418977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418881 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.418977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418905 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.418977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418940 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.418977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418958 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.418977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.418980 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config\") pod \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\" (UID: \"3ee8c4e7-fcbf-48d6-aa68-6294521aa485\") " Oct 30 18:32:35.419371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.419342 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:32:35.419464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.419410 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:35.419544 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.419521 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:35.419603 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.419526 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:32:35.421166 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.421112 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:35.421166 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.421131 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:32:35.421307 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.421213 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d" pod "3ee8c4e7-fcbf-48d6-aa68-6294521aa485" (UID: "3ee8c4e7-fcbf-48d6-aa68-6294521aa485"). InnerVolumeSpecName "kube-api-access-j9x5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:32:35.520374 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520346 5070 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-serving-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.520374 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520372 5070 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-service-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.520545 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520381 5070 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-config\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.520545 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520390 5070 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-console-oauth-config\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.520545 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520399 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9x5d\" (UniqueName: \"kubernetes.io/projected/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-kube-api-access-j9x5d\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.520545 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520407 5070 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-oauth-serving-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.520545 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.520416 5070 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ee8c4e7-fcbf-48d6-aa68-6294521aa485-tmp\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:32:35.533699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.533670 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:35.556484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.556451 5070 generic.go:356] "Generic (PLEG): container finished" podID="3ee8c4e7-fcbf-48d6-aa68-6294521aa485" containerID="9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4" exitCode=0 Oct 30 18:32:35.556587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.556491 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9444f6d9-fjllt" event={"ID":"3ee8c4e7-fcbf-48d6-aa68-6294521aa485","Type":"ContainerDied","Data":"9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4"} Oct 30 18:32:35.556587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.556539 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9444f6d9-fjllt" event={"ID":"3ee8c4e7-fcbf-48d6-aa68-6294521aa485","Type":"ContainerDied","Data":"3e2d2b1ade72d68a75268cb5a7e7a25bde001ce1e288ece942f9d5b079c9ff76"} Oct 30 18:32:35.556587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.556500 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9444f6d9-fjllt" Oct 30 18:32:35.556711 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.556554 5070 scope.go:122] "RemoveContainer" containerID="9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4" Oct 30 18:32:35.565847 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.565815 5070 scope.go:122] "RemoveContainer" containerID="9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4" Oct 30 18:32:35.566096 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:32:35.566076 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4\": container with ID starting with 9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4 not found: ID does not exist" containerID="9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4" Oct 30 18:32:35.566181 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.566108 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4"} err="failed to get container status \"9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4\": rpc error: code = NotFound desc = could not find container \"9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4\": container with ID starting with 9ae17e905e679e632b53c9c3a1bcec0c74b40ee099895ad124db4da76e3b86a4 not found: ID does not exist" Oct 30 18:32:35.608325 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.608251 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f9444f6d9-fjllt"] Oct 30 18:32:35.614090 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.614063 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f9444f6d9-fjllt"] Oct 30 18:32:35.774773 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.774740 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:35.935225 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:35.935194 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:37.030342 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:37.030309 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ee8c4e7-fcbf-48d6-aa68-6294521aa485" path="/var/lib/kubelet/pods/3ee8c4e7-fcbf-48d6-aa68-6294521aa485/volumes" Oct 30 18:32:38.942365 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:38.942335 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:39.094896 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:39.094863 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:43.979501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:43.979462 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:44.133784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:44.133740 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:45.219349 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:45.219312 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:45.368314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:45.368285 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:32:54.635198 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:54.635166 5070 generic.go:356] "Generic (PLEG): container finished" podID="30e186ca-854c-417a-96ed-3a660db8b8f4" containerID="0410e4b9880b8c4646d68d19f8319c518a17a6423241214e10ed01509a47ded4" exitCode=0 Oct 30 18:32:54.635198 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:54.635208 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" event={"ID":"30e186ca-854c-417a-96ed-3a660db8b8f4","Type":"ContainerDied","Data":"0410e4b9880b8c4646d68d19f8319c518a17a6423241214e10ed01509a47ded4"} Oct 30 18:32:54.635619 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:54.635494 5070 scope.go:122] "RemoveContainer" containerID="0410e4b9880b8c4646d68d19f8319c518a17a6423241214e10ed01509a47ded4" Oct 30 18:32:55.641458 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:55.641420 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-8k4t8" event={"ID":"30e186ca-854c-417a-96ed-3a660db8b8f4","Type":"ContainerStarted","Data":"6f87ad403d44046f0614381f8c1f545d5fd80171fa168c20c0e5d846061b29cd"} Oct 30 18:32:58.977105 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:32:58.977060 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:00.291924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:00.291866 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:03.832723 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:33:03.832684 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:33:08.693695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:08.693658 5070 generic.go:356] "Generic (PLEG): container finished" podID="89a201b9-118c-43aa-8be1-1329c22d9130" containerID="b7f88f1f21b509bf7d139a1a0c164e6ccbd975011a86c42cd8b5eb6a902d342b" exitCode=0 Oct 30 18:33:08.694105 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:08.693733 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" event={"ID":"89a201b9-118c-43aa-8be1-1329c22d9130","Type":"ContainerDied","Data":"b7f88f1f21b509bf7d139a1a0c164e6ccbd975011a86c42cd8b5eb6a902d342b"} Oct 30 18:33:09.847125 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.847101 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:33:09.996942 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.996850 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h\" (UniqueName: \"kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h\") pod \"89a201b9-118c-43aa-8be1-1329c22d9130\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " Oct 30 18:33:09.996942 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.996904 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle\") pod \"89a201b9-118c-43aa-8be1-1329c22d9130\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " Oct 30 18:33:09.996942 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.996942 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle\") pod \"89a201b9-118c-43aa-8be1-1329c22d9130\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " Oct 30 18:33:09.997188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.997042 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path\") pod \"89a201b9-118c-43aa-8be1-1329c22d9130\" (UID: \"89a201b9-118c-43aa-8be1-1329c22d9130\") " Oct 30 18:33:09.997346 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.997316 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle" pod "89a201b9-118c-43aa-8be1-1329c22d9130" (UID: "89a201b9-118c-43aa-8be1-1329c22d9130"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:33:09.997346 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.997339 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle" pod "89a201b9-118c-43aa-8be1-1329c22d9130" (UID: "89a201b9-118c-43aa-8be1-1329c22d9130"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:33:09.997503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.997377 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path" pod "89a201b9-118c-43aa-8be1-1329c22d9130" (UID: "89a201b9-118c-43aa-8be1-1329c22d9130"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:33:09.998946 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:09.998924 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h" pod "89a201b9-118c-43aa-8be1-1329c22d9130" (UID: "89a201b9-118c-43aa-8be1-1329c22d9130"). InnerVolumeSpecName "kube-api-access-qm92h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:33:10.098025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.097992 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm92h\" (UniqueName: \"kubernetes.io/projected/89a201b9-118c-43aa-8be1-1329c22d9130-kube-api-access-qm92h\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:33:10.098025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.098019 5070 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-trusted-ca-bundle\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:33:10.098025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.098029 5070 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89a201b9-118c-43aa-8be1-1329c22d9130-service-ca-bundle\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:33:10.098271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.098040 5070 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/89a201b9-118c-43aa-8be1-1329c22d9130-archives-path\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:33:10.702304 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.702271 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" Oct 30 18:33:10.702475 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.702278 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6jh2k-wg5tb" event={"ID":"89a201b9-118c-43aa-8be1-1329c22d9130","Type":"ContainerDied","Data":"00869ff1329ded85d7c920cfef789b1406c10eac025df27ddda9b1b96b3316e4"} Oct 30 18:33:10.702475 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:10.702396 5070 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00869ff1329ded85d7c920cfef789b1406c10eac025df27ddda9b1b96b3316e4" Oct 30 18:33:11.199318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:11.199208 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:11.359511 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:11.359442 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:13.184970 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:13.184934 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:30.155829 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:30.155792 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:30.322334 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:33:30.322298 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:33:33.831722 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:33:33.831683 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:34:03.832983 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:34:03.832899 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:34:13.323039 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:34:13.323002 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:34:14.328500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:34:14.328458 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:34:14.488003 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:34:14.487966 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:34:33.835298 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:34:33.835253 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:34:38.590472 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:34:38.590440 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:03.832504 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:35:03.832465 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:35:14.242434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:14.242402 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:14.430807 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:14.430769 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:14.618444 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:14.618414 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:15.947444 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:15.947412 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:16.097029 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:16.096997 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:17.285619 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:17.285584 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:17.518816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:17.518782 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:20.828836 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:20.828799 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:20.990778 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:20.990739 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:22.212393 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.212354 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:22.261134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261101 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd"] Oct 30 18:35:22.261605 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261583 5070 cpu_manager.go:414] "containerMap: removing container" podUID="89a201b9-118c-43aa-8be1-1329c22d9130" containerName="insights-gathering" Oct 30 18:35:22.261669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261610 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89a201b9-118c-43aa-8be1-1329c22d9130" containerName="insights-gathering" Oct 30 18:35:22.261669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261625 5070 cpu_manager.go:414] "containerMap: removing container" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerName="registry" Oct 30 18:35:22.261669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261636 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerName="registry" Oct 30 18:35:22.261669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261656 5070 cpu_manager.go:414] "containerMap: removing container" podUID="843291f5-2d22-4ba5-b99b-b499cca25268" containerName="console" Oct 30 18:35:22.261669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261666 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="843291f5-2d22-4ba5-b99b-b499cca25268" containerName="console" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261681 5070 cpu_manager.go:414] "containerMap: removing container" podUID="3ee8c4e7-fcbf-48d6-aa68-6294521aa485" containerName="console" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261689 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ee8c4e7-fcbf-48d6-aa68-6294521aa485" containerName="console" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261711 5070 cpu_manager.go:414] "containerMap: removing container" podUID="faa532be-e39c-4a3d-bf81-da68387b9703" containerName="insights-gathering" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261721 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="faa532be-e39c-4a3d-bf81-da68387b9703" containerName="insights-gathering" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261732 5070 cpu_manager.go:414] "containerMap: removing container" podUID="865d20d3-b981-4d85-a56b-4951ae770312" containerName="registry" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261740 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="865d20d3-b981-4d85-a56b-4951ae770312" containerName="registry" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261839 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e6926c1-1692-4111-85b6-319433b9bdfe" containerName="registry" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261852 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="843291f5-2d22-4ba5-b99b-b499cca25268" containerName="console" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261862 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ee8c4e7-fcbf-48d6-aa68-6294521aa485" containerName="console" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261872 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="faa532be-e39c-4a3d-bf81-da68387b9703" containerName="insights-gathering" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261881 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="865d20d3-b981-4d85-a56b-4951ae770312" containerName="registry" Oct 30 18:35:22.261956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.261894 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="89a201b9-118c-43aa-8be1-1329c22d9130" containerName="insights-gathering" Oct 30 18:35:22.264373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.264348 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.280543 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.280522 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-f4hrj\"" Oct 30 18:35:22.283986 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.283970 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 18:35:22.296229 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.296204 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd"] Oct 30 18:35:22.327347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327318 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327352 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a34c34-38fc-4dc4-a72b-79a371564b82-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327439 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327490 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327555 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327595 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsvkk\" (UniqueName: \"kubernetes.io/projected/92a34c34-38fc-4dc4-a72b-79a371564b82-kube-api-access-xsvkk\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327728 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327626 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327728 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327649 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.327728 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.327667 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.364317 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.364291 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:22.428291 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428263 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428409 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428295 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428409 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428319 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428428 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsvkk\" (UniqueName: \"kubernetes.io/projected/92a34c34-38fc-4dc4-a72b-79a371564b82-kube-api-access-xsvkk\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428487 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428516 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428532 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428570 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428594 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a34c34-38fc-4dc4-a72b-79a371564b82-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428666 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.428892 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.428728 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.429068 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.429046 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.429129 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.429064 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.429282 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.429265 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92a34c34-38fc-4dc4-a72b-79a371564b82-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.431116 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.431091 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.431288 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.431272 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.452453 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.452430 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92a34c34-38fc-4dc4-a72b-79a371564b82-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.452711 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.452692 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsvkk\" (UniqueName: \"kubernetes.io/projected/92a34c34-38fc-4dc4-a72b-79a371564b82-kube-api-access-xsvkk\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd\" (UID: \"92a34c34-38fc-4dc4-a72b-79a371564b82\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.516593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.516509 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:22.577046 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.577015 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:22.890161 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:22.890123 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd"] Oct 30 18:35:22.891418 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:35:22.891395 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a34c34_38fc_4dc4_a72b_79a371564b82.slice/crio-e88cb3e1babb5717a39ff3811c2732b6b1f57424cdc88e5e49779f35d0f1afb9 WatchSource:0}: Error finding container e88cb3e1babb5717a39ff3811c2732b6b1f57424cdc88e5e49779f35d0f1afb9: Status 404 returned error can't find the container with id e88cb3e1babb5717a39ff3811c2732b6b1f57424cdc88e5e49779f35d0f1afb9 Oct 30 18:35:23.216526 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:23.216442 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" event={"ID":"92a34c34-38fc-4dc4-a72b-79a371564b82","Type":"ContainerStarted","Data":"e88cb3e1babb5717a39ff3811c2732b6b1f57424cdc88e5e49779f35d0f1afb9"} Oct 30 18:35:25.399833 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:25.399798 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:35:25.400124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:25.399866 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:35:25.400124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:25.399896 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:35:25.488776 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:25.488734 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:26.231915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:26.231879 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" event={"ID":"92a34c34-38fc-4dc4-a72b-79a371564b82","Type":"ContainerStarted","Data":"0f197472f8f5dfc73cecc9d7c0f7e38651990f0c82a3aabc7667a476e6c1c275"} Oct 30 18:35:26.292806 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:26.292759 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" podStartSLOduration=1.786957747 podStartE2EDuration="4.292746775s" totalImagesPullingTime="2.505789028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:26.291070197 +0000 UTC m=+243.406204114" watchObservedRunningTime="2026-10-30 18:35:26.292746775 +0000 UTC m=+243.407880693" Oct 30 18:35:26.578110 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:26.578030 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:26.582726 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:26.582702 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:27.235727 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:27.235697 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:27.236593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:27.236574 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd" Oct 30 18:35:33.831292 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:35:33.831254 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:35:42.880359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:42.880324 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:51.634684 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:51.634647 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:51.784828 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:51.784795 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:57.763247 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:57.763210 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:58.089802 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.089722 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:58.714945 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.714908 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:58.818938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.818901 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874"] Oct 30 18:35:58.825714 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.825697 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.828439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.828413 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.828565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.828447 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.828565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.828466 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/267afe49-5c51-4731-b7a6-d2d14070d443-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.828565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.828528 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.828738 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.828572 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvs5p\" (UniqueName: \"kubernetes.io/projected/267afe49-5c51-4731-b7a6-d2d14070d443-kube-api-access-bvs5p\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.828738 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.828690 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/267afe49-5c51-4731-b7a6-d2d14070d443-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.848653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.848630 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 18:35:58.855416 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.855401 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-znpc4\"" Oct 30 18:35:58.863752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.863733 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Oct 30 18:35:58.863922 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.863905 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Oct 30 18:35:58.909220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.909199 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874"] Oct 30 18:35:58.928980 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.928954 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/267afe49-5c51-4731-b7a6-d2d14070d443-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.929085 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.929007 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.929085 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.929031 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.929085 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.929049 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/267afe49-5c51-4731-b7a6-d2d14070d443-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.929472 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.929405 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.929472 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.929444 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/267afe49-5c51-4731-b7a6-d2d14070d443-manifests\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.929645 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.929450 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvs5p\" (UniqueName: \"kubernetes.io/projected/267afe49-5c51-4731-b7a6-d2d14070d443-kube-api-access-bvs5p\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.931317 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.931275 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/267afe49-5c51-4731-b7a6-d2d14070d443-tmp\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.931656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.931638 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.931766 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.931669 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.949240 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.949222 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 18:35:58.949959 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.949941 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Oct 30 18:35:58.956752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.956731 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvs5p\" (UniqueName: \"kubernetes.io/projected/267afe49-5c51-4731-b7a6-d2d14070d443-kube-api-access-bvs5p\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:58.962518 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:58.962478 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/267afe49-5c51-4731-b7a6-d2d14070d443-metrics-cert\") pod \"opendatahub-operator-controller-manager-6cbccd5bdc-gb874\" (UID: \"267afe49-5c51-4731-b7a6-d2d14070d443\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:59.097359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:59.097263 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:59.140252 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:59.140218 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:35:59.314356 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:59.314320 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:59.496616 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:59.496583 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:35:59.504916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:59.504890 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874"] Oct 30 18:35:59.505122 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:35:59.505100 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267afe49_5c51_4731_b7a6_d2d14070d443.slice/crio-eca2b05e0216e93de4a2544721592c2f98c885da5b731dd6ee12ba5732590b22 WatchSource:0}: Error finding container eca2b05e0216e93de4a2544721592c2f98c885da5b731dd6ee12ba5732590b22: Status 404 returned error can't find the container with id eca2b05e0216e93de4a2544721592c2f98c885da5b731dd6ee12ba5732590b22 Oct 30 18:35:59.684466 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:35:59.684427 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:00.366436 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:00.366396 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" event={"ID":"267afe49-5c51-4731-b7a6-d2d14070d443","Type":"ContainerStarted","Data":"eca2b05e0216e93de4a2544721592c2f98c885da5b731dd6ee12ba5732590b22"} Oct 30 18:36:01.537259 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:01.537226 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:01.724974 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:01.724933 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:03.371956 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:03.371916 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:03.381231 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:03.381203 5070 generic.go:356] "Generic (PLEG): container finished" podID="267afe49-5c51-4731-b7a6-d2d14070d443" containerID="abc371972d8fa3c37a82051bf508b687014869b3aca475f13b7be04c23708bba" exitCode=0 Oct 30 18:36:03.381398 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:03.381264 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" event={"ID":"267afe49-5c51-4731-b7a6-d2d14070d443","Type":"ContainerDied","Data":"abc371972d8fa3c37a82051bf508b687014869b3aca475f13b7be04c23708bba"} Oct 30 18:36:03.831891 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:36:03.831854 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:36:04.387072 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.387030 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" event={"ID":"267afe49-5c51-4731-b7a6-d2d14070d443","Type":"ContainerStarted","Data":"195019809d592b6bc2787a0df5a12a67e797a25bf4c22601d58705051a95eb2c"} Oct 30 18:36:04.387513 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.387105 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:36:04.477382 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.477323 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" podStartSLOduration=2.823188178 podStartE2EDuration="6.477309914s" totalImagesPullingTime="3.221471682s" totalInitContainerRuntime="432.650054ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:04.463895994 +0000 UTC m=+281.579029938" watchObservedRunningTime="2026-10-30 18:36:04.477309914 +0000 UTC m=+281.592443825" Oct 30 18:36:04.832345 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.832263 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986"] Oct 30 18:36:04.835698 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.835678 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:04.853274 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.853252 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 18:36:04.853370 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.853345 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zzf79\"" Oct 30 18:36:04.864015 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.863992 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Oct 30 18:36:04.864268 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.864248 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 18:36:04.866879 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.866856 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986"] Oct 30 18:36:04.882922 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.882901 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbnf6\" (UniqueName: \"kubernetes.io/projected/22e1e7b3-20c9-4065-b00a-ea6842e637cb-kube-api-access-vbnf6\") pod \"dns-operator-controller-manager-8557cbdb84-bk986\" (UID: \"22e1e7b3-20c9-4065-b00a-ea6842e637cb\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:04.983476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:04.983443 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbnf6\" (UniqueName: \"kubernetes.io/projected/22e1e7b3-20c9-4065-b00a-ea6842e637cb-kube-api-access-vbnf6\") pod \"dns-operator-controller-manager-8557cbdb84-bk986\" (UID: \"22e1e7b3-20c9-4065-b00a-ea6842e637cb\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:05.008113 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:05.008076 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbnf6\" (UniqueName: \"kubernetes.io/projected/22e1e7b3-20c9-4065-b00a-ea6842e637cb-kube-api-access-vbnf6\") pod \"dns-operator-controller-manager-8557cbdb84-bk986\" (UID: \"22e1e7b3-20c9-4065-b00a-ea6842e637cb\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:05.048260 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:05.048228 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:05.147921 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:05.147890 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:05.450643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:05.450612 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986"] Oct 30 18:36:05.451388 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:05.451358 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e1e7b3_20c9_4065_b00a_ea6842e637cb.slice/crio-8da8522e3e78846b7aac400d03ee4ec5abfa494090339e9f586f029cbe68395c WatchSource:0}: Error finding container 8da8522e3e78846b7aac400d03ee4ec5abfa494090339e9f586f029cbe68395c: Status 404 returned error can't find the container with id 8da8522e3e78846b7aac400d03ee4ec5abfa494090339e9f586f029cbe68395c Oct 30 18:36:06.396435 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:06.396394 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" event={"ID":"22e1e7b3-20c9-4065-b00a-ea6842e637cb","Type":"ContainerStarted","Data":"8da8522e3e78846b7aac400d03ee4ec5abfa494090339e9f586f029cbe68395c"} Oct 30 18:36:06.556024 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:06.555977 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:08.411177 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.411125 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" event={"ID":"22e1e7b3-20c9-4065-b00a-ea6842e637cb","Type":"ContainerStarted","Data":"e194f70a453edd873ea46970a2f30fbecb3540908cf4b6815f6ffba465aa2200"} Oct 30 18:36:08.411622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.411267 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:08.457777 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.457719 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" podStartSLOduration=1.6637769009999999 podStartE2EDuration="4.457704695s" totalImagesPullingTime="2.793927794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:08.456108619 +0000 UTC m=+285.571242536" watchObservedRunningTime="2026-10-30 18:36:08.457704695 +0000 UTC m=+285.572838640" Oct 30 18:36:08.694626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.694586 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c"] Oct 30 18:36:08.698084 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.698052 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:08.723489 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.723450 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-km4t2\"" Oct 30 18:36:08.724964 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.724942 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c"] Oct 30 18:36:08.819135 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.819094 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmgwm\" (UniqueName: \"kubernetes.io/projected/568c6bf1-8dd1-4aeb-a18d-cebf58d57a56-kube-api-access-dmgwm\") pod \"limitador-operator-controller-manager-5778bb4449-xwz9c\" (UID: \"568c6bf1-8dd1-4aeb-a18d-cebf58d57a56\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:08.919999 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.919956 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmgwm\" (UniqueName: \"kubernetes.io/projected/568c6bf1-8dd1-4aeb-a18d-cebf58d57a56-kube-api-access-dmgwm\") pod \"limitador-operator-controller-manager-5778bb4449-xwz9c\" (UID: \"568c6bf1-8dd1-4aeb-a18d-cebf58d57a56\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:08.946006 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.945933 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmgwm\" (UniqueName: \"kubernetes.io/projected/568c6bf1-8dd1-4aeb-a18d-cebf58d57a56-kube-api-access-dmgwm\") pod \"limitador-operator-controller-manager-5778bb4449-xwz9c\" (UID: \"568c6bf1-8dd1-4aeb-a18d-cebf58d57a56\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:08.958238 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:08.958208 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:09.011125 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:09.011090 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:09.125439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:09.125404 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:09.311572 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:09.311473 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c"] Oct 30 18:36:09.317665 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:09.317631 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568c6bf1_8dd1_4aeb_a18d_cebf58d57a56.slice/crio-b3f3211fd5802e987b2e94a73ba8ab52c8a2979cffbd83d5c767856ac00f1d7c WatchSource:0}: Error finding container b3f3211fd5802e987b2e94a73ba8ab52c8a2979cffbd83d5c767856ac00f1d7c: Status 404 returned error can't find the container with id b3f3211fd5802e987b2e94a73ba8ab52c8a2979cffbd83d5c767856ac00f1d7c Oct 30 18:36:09.417329 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:09.416986 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" event={"ID":"568c6bf1-8dd1-4aeb-a18d-cebf58d57a56","Type":"ContainerStarted","Data":"b3f3211fd5802e987b2e94a73ba8ab52c8a2979cffbd83d5c767856ac00f1d7c"} Oct 30 18:36:09.439637 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:09.439601 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:09.633357 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:09.633322 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:10.581358 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:10.581322 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:11.350450 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:11.350419 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:11.433203 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:11.433081 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" event={"ID":"568c6bf1-8dd1-4aeb-a18d-cebf58d57a56","Type":"ContainerStarted","Data":"b11396a6272f97b4dfe9fb22abcdd47c73038280957ef0aa8dcbeecf3a7f454c"} Oct 30 18:36:11.433388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:11.433231 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:11.493333 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:11.493279 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" podStartSLOduration=1.685241881 podStartE2EDuration="3.493265125s" totalImagesPullingTime="1.808023244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:11.492112708 +0000 UTC m=+288.607246624" watchObservedRunningTime="2026-10-30 18:36:11.493265125 +0000 UTC m=+288.608399043" Oct 30 18:36:11.987891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:11.987851 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:12.922130 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:12.922095 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:13.677139 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:13.677105 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:13.882046 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:13.882015 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:14.578809 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:14.578780 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:15.393500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:15.393470 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6cbccd5bdc-gb874" Oct 30 18:36:15.615797 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:15.615768 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:16.114359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:16.114328 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:17.343763 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:17.343728 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:17.650228 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:17.650196 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:18.206195 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:18.206162 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:18.620108 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:18.620076 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:18.899095 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:18.899013 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:19.419721 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:19.419692 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-bk986" Oct 30 18:36:21.398997 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.398956 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-trgmc"] Oct 30 18:36:21.402547 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.402521 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:21.423812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.423789 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-566mk\"" Oct 30 18:36:21.437490 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.437466 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-trgmc"] Oct 30 18:36:21.474083 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.474046 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:21.533470 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.533389 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zqc8\" (UniqueName: \"kubernetes.io/projected/cbe4deb1-4c90-4cd7-a540-c0bcff310c70-kube-api-access-9zqc8\") pod \"authorino-operator-7b7bb4bc77-trgmc\" (UID: \"cbe4deb1-4c90-4cd7-a540-c0bcff310c70\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:21.634441 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.634404 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zqc8\" (UniqueName: \"kubernetes.io/projected/cbe4deb1-4c90-4cd7-a540-c0bcff310c70-kube-api-access-9zqc8\") pod \"authorino-operator-7b7bb4bc77-trgmc\" (UID: \"cbe4deb1-4c90-4cd7-a540-c0bcff310c70\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:21.664662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.664633 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zqc8\" (UniqueName: \"kubernetes.io/projected/cbe4deb1-4c90-4cd7-a540-c0bcff310c70-kube-api-access-9zqc8\") pod \"authorino-operator-7b7bb4bc77-trgmc\" (UID: \"cbe4deb1-4c90-4cd7-a540-c0bcff310c70\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:21.704061 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.704032 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:21.714963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:21.714942 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:22.015283 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:22.015260 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-trgmc"] Oct 30 18:36:22.017028 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:22.016995 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe4deb1_4c90_4cd7_a540_c0bcff310c70.slice/crio-55fbe4ef3a6242407bf73d2d35abc8f9fecdb727dbe7d1a02b1b8fac692d2653 WatchSource:0}: Error finding container 55fbe4ef3a6242407bf73d2d35abc8f9fecdb727dbe7d1a02b1b8fac692d2653: Status 404 returned error can't find the container with id 55fbe4ef3a6242407bf73d2d35abc8f9fecdb727dbe7d1a02b1b8fac692d2653 Oct 30 18:36:22.440942 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:22.440915 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-xwz9c" Oct 30 18:36:22.482108 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:22.482073 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" event={"ID":"cbe4deb1-4c90-4cd7-a540-c0bcff310c70","Type":"ContainerStarted","Data":"55fbe4ef3a6242407bf73d2d35abc8f9fecdb727dbe7d1a02b1b8fac692d2653"} Oct 30 18:36:23.055813 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.055779 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:36:23.056544 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.056525 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:36:23.061611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.061589 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:36:23.061918 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.061903 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:36:23.488259 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.488229 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" event={"ID":"cbe4deb1-4c90-4cd7-a540-c0bcff310c70","Type":"ContainerStarted","Data":"63ddb16d0ec270fb21e0b67412ab71c2e4d417e9d95a0514578cf3a6c564e770"} Oct 30 18:36:23.488607 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.488353 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:23.547734 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.547688 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" podStartSLOduration=1.166633167 podStartE2EDuration="2.547675863s" totalImagesPullingTime="1.381042696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:23.546588494 +0000 UTC m=+300.661722412" watchObservedRunningTime="2026-10-30 18:36:23.547675863 +0000 UTC m=+300.662809780" Oct 30 18:36:23.632738 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.632702 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:23.813241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.813162 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:23.984262 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:23.984226 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:24.158253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:24.158222 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:24.314851 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:24.314822 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:25.215019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:25.214984 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:25.395466 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:25.395437 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:27.396051 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:27.396003 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:27.567626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:27.567581 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:27.756754 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:27.756672 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:27.909179 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:27.909131 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:28.119308 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.119273 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:28.304776 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.304749 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:28.309331 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.309312 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.339254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.339226 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kmnd7\"" Oct 30 18:36:28.352102 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.352073 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 30 18:36:28.353756 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.353731 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:28.358105 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.358069 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:28.393525 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.392943 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2smr2\" (UniqueName: \"kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2\") pod \"limitador-limitador-74dfcfb894-6b2gq\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.393871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.393843 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file\") pod \"limitador-limitador-74dfcfb894-6b2gq\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.396227 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.396204 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4"] Oct 30 18:36:28.404777 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.404757 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.424097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.424074 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4"] Oct 30 18:36:28.439597 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.439574 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wjsp4\"" Oct 30 18:36:28.473314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.473279 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:28.495439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495409 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2smr2\" (UniqueName: \"kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2\") pod \"limitador-limitador-74dfcfb894-6b2gq\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.495606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495468 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495560 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495689 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495646 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979f3204-6db1-4a7d-9863-7b11c566154e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495696 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file\") pod \"limitador-limitador-74dfcfb894-6b2gq\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.495778 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495755 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495794 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979f3204-6db1-4a7d-9863-7b11c566154e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495854 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495819 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzw5n\" (UniqueName: \"kubernetes.io/projected/979f3204-6db1-4a7d-9863-7b11c566154e-kube-api-access-lzw5n\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495886 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495873 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495919 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.495980 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.495968 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979f3204-6db1-4a7d-9863-7b11c566154e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.496341 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.496325 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file\") pod \"limitador-limitador-74dfcfb894-6b2gq\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.520276 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.520244 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2smr2\" (UniqueName: \"kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2\") pod \"limitador-limitador-74dfcfb894-6b2gq\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.520459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.520422 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:28.597017 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.596988 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979f3204-6db1-4a7d-9863-7b11c566154e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597267 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597048 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597267 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597167 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597267 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597190 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979f3204-6db1-4a7d-9863-7b11c566154e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597267 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597250 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597273 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979f3204-6db1-4a7d-9863-7b11c566154e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597290 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzw5n\" (UniqueName: \"kubernetes.io/projected/979f3204-6db1-4a7d-9863-7b11c566154e-kube-api-access-lzw5n\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597320 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597658 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597346 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597770 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597726 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597900 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.597992 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597912 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.598050 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.597998 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/979f3204-6db1-4a7d-9863-7b11c566154e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.598114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.598062 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.599614 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.599592 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/979f3204-6db1-4a7d-9863-7b11c566154e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.599709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.599658 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/979f3204-6db1-4a7d-9863-7b11c566154e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.621425 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.621400 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:28.627007 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.626980 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/979f3204-6db1-4a7d-9863-7b11c566154e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.627126 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.627081 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzw5n\" (UniqueName: \"kubernetes.io/projected/979f3204-6db1-4a7d-9863-7b11c566154e-kube-api-access-lzw5n\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4\" (UID: \"979f3204-6db1-4a7d-9863-7b11c566154e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.686241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.686211 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:28.720935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.720903 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:28.933943 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.933917 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:28.935482 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:28.935457 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4d6a30_c6ca_4f78_a65f_5f17bcb155a3.slice/crio-62bb26a6d15e5f40d99495e4de1ca4adc27e7b0e3405897c88bbbad59cd7a51c WatchSource:0}: Error finding container 62bb26a6d15e5f40d99495e4de1ca4adc27e7b0e3405897c88bbbad59cd7a51c: Status 404 returned error can't find the container with id 62bb26a6d15e5f40d99495e4de1ca4adc27e7b0e3405897c88bbbad59cd7a51c Oct 30 18:36:28.937443 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:28.937426 5070 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:36:29.031609 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:29.031570 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979f3204_6db1_4a7d_9863_7b11c566154e.slice/crio-64bc6c3f696bfaf500de838e60d6fdd0010d475f9fe97a8e868857262d580d14 WatchSource:0}: Error finding container 64bc6c3f696bfaf500de838e60d6fdd0010d475f9fe97a8e868857262d580d14: Status 404 returned error can't find the container with id 64bc6c3f696bfaf500de838e60d6fdd0010d475f9fe97a8e868857262d580d14 Oct 30 18:36:29.032085 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.032065 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4"] Oct 30 18:36:29.033738 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.033705 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:36:29.033852 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.033769 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:36:29.033852 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.033805 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:36:29.328220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.328137 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd"] Oct 30 18:36:29.331639 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.331617 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.355279 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.355257 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd"] Oct 30 18:36:29.355485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.355467 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 18:36:29.355645 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.355591 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 18:36:29.355696 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.355659 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nbzvt\"" Oct 30 18:36:29.363571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.363544 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:29.406040 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.406005 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wq5qd\" (UID: \"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.406420 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.406046 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8mhz\" (UniqueName: \"kubernetes.io/projected/ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1-kube-api-access-j8mhz\") pod \"cert-manager-cainjector-5c57bbd4fc-wq5qd\" (UID: \"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.507496 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.507461 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wq5qd\" (UID: \"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.507704 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.507518 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8mhz\" (UniqueName: \"kubernetes.io/projected/ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1-kube-api-access-j8mhz\") pod \"cert-manager-cainjector-5c57bbd4fc-wq5qd\" (UID: \"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.516806 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.516773 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" event={"ID":"979f3204-6db1-4a7d-9863-7b11c566154e","Type":"ContainerStarted","Data":"e7e97c50938629e8a15fa450fcfc94d83f73af79e2aac819f62704a617b96676"} Oct 30 18:36:29.516951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.516832 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" event={"ID":"979f3204-6db1-4a7d-9863-7b11c566154e","Type":"ContainerStarted","Data":"64bc6c3f696bfaf500de838e60d6fdd0010d475f9fe97a8e868857262d580d14"} Oct 30 18:36:29.518071 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.518041 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" event={"ID":"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3","Type":"ContainerStarted","Data":"62bb26a6d15e5f40d99495e4de1ca4adc27e7b0e3405897c88bbbad59cd7a51c"} Oct 30 18:36:29.535235 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.535206 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8mhz\" (UniqueName: \"kubernetes.io/projected/ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1-kube-api-access-j8mhz\") pod \"cert-manager-cainjector-5c57bbd4fc-wq5qd\" (UID: \"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.535378 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.535363 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-wq5qd\" (UID: \"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.558341 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.558312 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:29.572103 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.572061 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" podStartSLOduration=1.572048105 podStartE2EDuration="1.572048105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:29.570574855 +0000 UTC m=+306.685708773" watchObservedRunningTime="2026-10-30 18:36:29.572048105 +0000 UTC m=+306.687182022" Oct 30 18:36:29.652616 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.652581 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" Oct 30 18:36:29.725841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.721629 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:29.849879 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.849833 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:29.961638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:29.961610 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd"] Oct 30 18:36:29.963282 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:29.963252 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac8ada95_cc5f_42fe_8bbe_0e3224de5ae1.slice/crio-3d899d9f343d1107d976ff3a64ae5efcc65c32e4cceb6083c52ab32ae766f6fc WatchSource:0}: Error finding container 3d899d9f343d1107d976ff3a64ae5efcc65c32e4cceb6083c52ab32ae766f6fc: Status 404 returned error can't find the container with id 3d899d9f343d1107d976ff3a64ae5efcc65c32e4cceb6083c52ab32ae766f6fc Oct 30 18:36:30.530477 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:30.530360 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" event={"ID":"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3","Type":"ContainerStarted","Data":"bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d"} Oct 30 18:36:30.530477 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:30.530441 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:30.532448 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:30.532408 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" event={"ID":"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1","Type":"ContainerStarted","Data":"3d899d9f343d1107d976ff3a64ae5efcc65c32e4cceb6083c52ab32ae766f6fc"} Oct 30 18:36:30.582983 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:30.582924 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" podStartSLOduration=1.38312265 podStartE2EDuration="2.5829074s" totalImagesPullingTime="1.19978475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:30.582432766 +0000 UTC m=+307.697566703" watchObservedRunningTime="2026-10-30 18:36:30.5829074 +0000 UTC m=+307.698041319" Oct 30 18:36:30.728276 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:30.728227 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:31.181605 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.181563 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:31.559522 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.559441 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:31.560626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.560597 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4" Oct 30 18:36:31.703093 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.703044 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:31.869263 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.869232 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:31.917982 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.917944 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-nkx9x"] Oct 30 18:36:31.921419 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.921397 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:31.946831 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.946796 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-nkx9x"] Oct 30 18:36:31.992833 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:31.992805 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2plmc\"" Oct 30 18:36:32.036048 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.036015 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knb5p\" (UniqueName: \"kubernetes.io/projected/1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9-kube-api-access-knb5p\") pod \"cert-manager-597b95cb84-nkx9x\" (UID: \"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9\") " pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.036231 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.036070 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9-bound-sa-token\") pod \"cert-manager-597b95cb84-nkx9x\" (UID: \"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9\") " pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.085025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.084980 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:32.137594 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.137507 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knb5p\" (UniqueName: \"kubernetes.io/projected/1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9-kube-api-access-knb5p\") pod \"cert-manager-597b95cb84-nkx9x\" (UID: \"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9\") " pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.137594 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.137574 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9-bound-sa-token\") pod \"cert-manager-597b95cb84-nkx9x\" (UID: \"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9\") " pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.163126 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.163086 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knb5p\" (UniqueName: \"kubernetes.io/projected/1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9-kube-api-access-knb5p\") pod \"cert-manager-597b95cb84-nkx9x\" (UID: \"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9\") " pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.170627 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.170596 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9-bound-sa-token\") pod \"cert-manager-597b95cb84-nkx9x\" (UID: \"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9\") " pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.233874 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.233835 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-nkx9x" Oct 30 18:36:32.536971 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.536944 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-nkx9x"] Oct 30 18:36:32.538667 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:32.538636 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e6bdef6_7869_49a0_ac7d_8fb9b50ab5b9.slice/crio-5e9a321a1927f9759e7caa05093c4cd2289ea4bb2d53b1cb8c4bf7d11b7b25c8 WatchSource:0}: Error finding container 5e9a321a1927f9759e7caa05093c4cd2289ea4bb2d53b1cb8c4bf7d11b7b25c8: Status 404 returned error can't find the container with id 5e9a321a1927f9759e7caa05093c4cd2289ea4bb2d53b1cb8c4bf7d11b7b25c8 Oct 30 18:36:32.564436 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:32.564402 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-nkx9x" event={"ID":"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9","Type":"ContainerStarted","Data":"5e9a321a1927f9759e7caa05093c4cd2289ea4bb2d53b1cb8c4bf7d11b7b25c8"} Oct 30 18:36:33.571172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:33.571043 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" event={"ID":"ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1","Type":"ContainerStarted","Data":"d9618e2de56e86311b4d5acaa884b80d367fb02225f4a5a7d5a8647ef0b7c37d"} Oct 30 18:36:33.572595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:33.572570 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-nkx9x" event={"ID":"1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9","Type":"ContainerStarted","Data":"268cdbdefb083121ef0b9be62b1b9fe65f43803789ab67aa330547824eccab25"} Oct 30 18:36:33.612554 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:33.612504 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-wq5qd" podStartSLOduration=1.659552385 podStartE2EDuration="4.61249164s" totalImagesPullingTime="2.952939255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:33.611880701 +0000 UTC m=+310.727014618" watchObservedRunningTime="2026-10-30 18:36:33.61249164 +0000 UTC m=+310.727625556" Oct 30 18:36:33.656752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:33.656700 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-nkx9x" podStartSLOduration=2.039999943 podStartE2EDuration="2.656686803s" totalImagesPullingTime="616.68686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:33.656314307 +0000 UTC m=+310.771448248" watchObservedRunningTime="2026-10-30 18:36:33.656686803 +0000 UTC m=+310.771820720" Oct 30 18:36:33.834607 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:36:33.834524 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:36:34.021032 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:34.020997 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:34.199816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:34.199783 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:34.495766 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:34.495692 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-trgmc" Oct 30 18:36:35.498097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:35.498058 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:38.398760 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:38.398725 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:38.548793 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:38.548763 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:40.578160 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:40.578110 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:40.735633 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:40.735600 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:41.559917 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:41.559887 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:43.164638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:43.164604 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:43.356530 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:43.356487 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:43.691025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:43.690992 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:43.691247 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:43.691211 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" podUID="2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" containerName="limitador" containerID="cri-o://bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d" gracePeriod=30 Oct 30 18:36:44.242346 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.242314 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:44.348768 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.348672 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file\" (UniqueName: \"kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file\") pod \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " Oct 30 18:36:44.348951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.348835 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2\" (UniqueName: \"kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2\") pod \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\" (UID: \"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3\") " Oct 30 18:36:44.349064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.349038 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file" pod "2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" (UID: "2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:36:44.351060 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.351038 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2" pod "2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" (UID: "2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3"). InnerVolumeSpecName "kube-api-access-2smr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:36:44.449528 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.449482 5070 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-config-file\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:36:44.449528 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.449527 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2smr2\" (UniqueName: \"kubernetes.io/projected/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3-kube-api-access-2smr2\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:36:44.620972 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.620934 5070 generic.go:356] "Generic (PLEG): container finished" podID="2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" containerID="bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d" exitCode=0 Oct 30 18:36:44.621164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.620990 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" event={"ID":"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3","Type":"ContainerDied","Data":"bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d"} Oct 30 18:36:44.621164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.621008 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" Oct 30 18:36:44.621164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.621037 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-6b2gq" event={"ID":"2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3","Type":"ContainerDied","Data":"62bb26a6d15e5f40d99495e4de1ca4adc27e7b0e3405897c88bbbad59cd7a51c"} Oct 30 18:36:44.621164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.621063 5070 scope.go:122] "RemoveContainer" containerID="bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d" Oct 30 18:36:44.629583 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.629563 5070 scope.go:122] "RemoveContainer" containerID="bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d" Oct 30 18:36:44.629878 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:36:44.629857 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d\": container with ID starting with bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d not found: ID does not exist" containerID="bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d" Oct 30 18:36:44.629925 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.629889 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d"} err="failed to get container status \"bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d\": rpc error: code = NotFound desc = could not find container \"bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d\": container with ID starting with bb629557e8484d0752e4f708d3910c26356f8f6ad9c6c6c8092d380d7061ed0d not found: ID does not exist" Oct 30 18:36:44.678534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.678499 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:44.691422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:44.691381 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-6b2gq"] Oct 30 18:36:45.031796 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:45.031718 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" path="/var/lib/kubelet/pods/2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3/volumes" Oct 30 18:36:46.365980 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:46.365937 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:46.523104 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:46.523073 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:50.131014 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.130965 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:50.323212 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.323100 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:50.891663 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.891629 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv"] Oct 30 18:36:50.892049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.892036 5070 cpu_manager.go:414] "containerMap: removing container" podUID="2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" containerName="limitador" Oct 30 18:36:50.892101 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.892051 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" containerName="limitador" Oct 30 18:36:50.892135 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.892115 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a4d6a30-c6ca-4f78-a65f-5f17bcb155a3" containerName="limitador" Oct 30 18:36:50.894683 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.894667 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:50.901078 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.900833 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck2kv\" (UniqueName: \"kubernetes.io/projected/a0adcbf7-e3ee-47f4-8b85-b092efc0df10-kube-api-access-ck2kv\") pod \"postgres-7cdbf6dc44-xgfxv\" (UID: \"a0adcbf7-e3ee-47f4-8b85-b092efc0df10\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:50.901078 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.900929 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a0adcbf7-e3ee-47f4-8b85-b092efc0df10-data\") pod \"postgres-7cdbf6dc44-xgfxv\" (UID: \"a0adcbf7-e3ee-47f4-8b85-b092efc0df10\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:50.914904 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.914875 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv"] Oct 30 18:36:50.920850 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.920826 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Oct 30 18:36:50.926310 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.926286 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:36:50.926422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.926340 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4kxxq\"" Oct 30 18:36:50.926422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:50.926343 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:36:51.001924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.001890 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck2kv\" (UniqueName: \"kubernetes.io/projected/a0adcbf7-e3ee-47f4-8b85-b092efc0df10-kube-api-access-ck2kv\") pod \"postgres-7cdbf6dc44-xgfxv\" (UID: \"a0adcbf7-e3ee-47f4-8b85-b092efc0df10\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:51.002118 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.001942 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a0adcbf7-e3ee-47f4-8b85-b092efc0df10-data\") pod \"postgres-7cdbf6dc44-xgfxv\" (UID: \"a0adcbf7-e3ee-47f4-8b85-b092efc0df10\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:51.002429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.002408 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a0adcbf7-e3ee-47f4-8b85-b092efc0df10-data\") pod \"postgres-7cdbf6dc44-xgfxv\" (UID: \"a0adcbf7-e3ee-47f4-8b85-b092efc0df10\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:51.022982 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.022950 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:51.027997 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.027970 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck2kv\" (UniqueName: \"kubernetes.io/projected/a0adcbf7-e3ee-47f4-8b85-b092efc0df10-kube-api-access-ck2kv\") pod \"postgres-7cdbf6dc44-xgfxv\" (UID: \"a0adcbf7-e3ee-47f4-8b85-b092efc0df10\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:51.201022 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.200944 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:51.206016 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.205996 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:51.365412 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.365380 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:51.509681 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.509650 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv"] Oct 30 18:36:51.510926 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:51.510899 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0adcbf7_e3ee_47f4_8b85_b092efc0df10.slice/crio-18447edfa4ce499b799d582c391f7024aa98586e4880578d18fc5557e050e3cb WatchSource:0}: Error finding container 18447edfa4ce499b799d582c391f7024aa98586e4880578d18fc5557e050e3cb: Status 404 returned error can't find the container with id 18447edfa4ce499b799d582c391f7024aa98586e4880578d18fc5557e050e3cb Oct 30 18:36:51.536646 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.536615 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:51.655318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.655285 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" event={"ID":"a0adcbf7-e3ee-47f4-8b85-b092efc0df10","Type":"ContainerStarted","Data":"18447edfa4ce499b799d582c391f7024aa98586e4880578d18fc5557e050e3cb"} Oct 30 18:36:51.706270 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.706238 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:51.940109 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:51.940075 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:54.391800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:54.391762 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:54.597898 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:54.597850 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:55.875063 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:55.875028 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:56.039087 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:56.039052 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:57.690435 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:57.690393 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" event={"ID":"a0adcbf7-e3ee-47f4-8b85-b092efc0df10","Type":"ContainerStarted","Data":"bd95e7b1fd02035b9dbb40243d579385f19b0d8ab159ce3892829bd2b7dfc57b"} Oct 30 18:36:57.690828 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:57.690517 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:36:57.733751 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:57.733684 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" podStartSLOduration=2.5738563020000003 podStartE2EDuration="7.733663685s" totalImagesPullingTime="5.159807383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:57.731590311 +0000 UTC m=+334.846724230" watchObservedRunningTime="2026-10-30 18:36:57.733663685 +0000 UTC m=+334.848797603" Oct 30 18:36:58.673566 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:58.673523 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:58.818930 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:58.818899 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:59.345110 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.345071 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh"] Oct 30 18:36:59.348991 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.348969 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.380881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.380838 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-v7bvd\"" Oct 30 18:36:59.380881 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.380838 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\"" Oct 30 18:36:59.381084 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.380929 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\"" Oct 30 18:36:59.381508 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.381486 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa302d0e-a330-40b7-bd14-6bd262531f62-cert\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.381599 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.381521 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb95q\" (UniqueName: \"kubernetes.io/projected/fa302d0e-a330-40b7-bd14-6bd262531f62-kube-api-access-kb95q\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.381599 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.381564 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa302d0e-a330-40b7-bd14-6bd262531f62-metrics-certs\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.381689 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.381633 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Oct 30 18:36:59.381778 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.381755 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fa302d0e-a330-40b7-bd14-6bd262531f62-manager-config\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.383563 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.383542 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh"] Oct 30 18:36:59.393587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.393559 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\"" Oct 30 18:36:59.393587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.393557 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:36:59.483008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.482975 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fa302d0e-a330-40b7-bd14-6bd262531f62-manager-config\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.483212 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.483019 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa302d0e-a330-40b7-bd14-6bd262531f62-cert\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.483212 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.483048 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb95q\" (UniqueName: \"kubernetes.io/projected/fa302d0e-a330-40b7-bd14-6bd262531f62-kube-api-access-kb95q\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.483212 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.483084 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa302d0e-a330-40b7-bd14-6bd262531f62-metrics-certs\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.483803 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.483775 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fa302d0e-a330-40b7-bd14-6bd262531f62-manager-config\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.485800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.485775 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa302d0e-a330-40b7-bd14-6bd262531f62-cert\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.485800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.485790 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa302d0e-a330-40b7-bd14-6bd262531f62-metrics-certs\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.507542 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.507505 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb95q\" (UniqueName: \"kubernetes.io/projected/fa302d0e-a330-40b7-bd14-6bd262531f62-kube-api-access-kb95q\") pod \"jobset-controller-manager-585f46ff54-gstjh\" (UID: \"fa302d0e-a330-40b7-bd14-6bd262531f62\") " pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.661827 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.661786 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:36:59.671801 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.671770 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:59.902533 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.902492 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:36:59.964765 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:36:59.964741 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh"] Oct 30 18:36:59.966843 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:36:59.966806 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa302d0e_a330_40b7_bd14_6bd262531f62.slice/crio-a046e6d35b7c6eb4fadea19a724cad80884abbfb9b2c089dadcb82e0a7065141 WatchSource:0}: Error finding container a046e6d35b7c6eb4fadea19a724cad80884abbfb9b2c089dadcb82e0a7065141: Status 404 returned error can't find the container with id a046e6d35b7c6eb4fadea19a724cad80884abbfb9b2c089dadcb82e0a7065141 Oct 30 18:37:00.706515 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:00.706477 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" event={"ID":"fa302d0e-a330-40b7-bd14-6bd262531f62","Type":"ContainerStarted","Data":"a046e6d35b7c6eb4fadea19a724cad80884abbfb9b2c089dadcb82e0a7065141"} Oct 30 18:37:03.722902 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:03.722865 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" event={"ID":"fa302d0e-a330-40b7-bd14-6bd262531f62","Type":"ContainerStarted","Data":"20d1e7283d0c06610051604f6001638d01712ae8fb0634b1974908af313d4ef5"} Oct 30 18:37:03.723397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:03.722946 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:37:03.724624 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:03.724604 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xgfxv" Oct 30 18:37:03.774855 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:03.774797 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" podStartSLOduration=1.886144482 podStartE2EDuration="4.774782345s" totalImagesPullingTime="2.888637863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:03.773736952 +0000 UTC m=+340.888870869" watchObservedRunningTime="2026-10-30 18:37:03.774782345 +0000 UTC m=+340.889916262" Oct 30 18:37:03.833126 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:37:03.833083 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:37:11.297126 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:11.297074 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:11.451648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:11.451599 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:11.617890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:11.617853 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:14.519390 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:14.519361 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:14.676662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:14.676623 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:14.732782 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:14.732751 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-585f46ff54-gstjh" Oct 30 18:37:15.717660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:15.717632 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:16.102200 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:16.102109 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:16.292297 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:16.292263 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:19.802068 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:19.802031 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:33.833629 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:37:33.833591 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:37:35.541223 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:35.541192 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:35.735307 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:35.735274 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:35.999340 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:35.999307 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:36.227752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:36.227719 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:36.640344 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:36.640301 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:39.457642 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:39.457610 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:39.657756 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:39.657720 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:39.846456 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:39.846380 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:40.053705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:40.053669 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:45.575062 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:45.575027 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:45.891672 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:45.891633 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:47.136482 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:47.136446 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:47.325075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:47.325038 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:47.540612 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:47.540528 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:57.767070 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:57.767035 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:37:57.919650 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:37:57.919619 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:03.831669 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:38:03.831632 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:38:15.866384 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:38:15.866342 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:20.002462 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:38:20.002420 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:33.833124 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:38:33.833085 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:39:03.831492 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:39:03.831449 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:39:16.029905 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:16.029868 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:16.205984 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:16.205948 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:16.362640 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:16.362554 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:20.166531 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:20.166493 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:20.596580 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:20.596504 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:20.802951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:20.802916 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:33.831974 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:39:33.831927 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:39:44.287401 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:44.287323 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:44.863823 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:44.863785 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:46.199435 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:46.199399 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:46.471103 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:39:46.471029 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:01.055318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:01.055281 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:03.832945 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:40:03.832910 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:40:05.253805 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.253763 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p"] Oct 30 18:40:05.258107 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.258084 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.277438 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.277417 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Oct 30 18:40:05.286593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.286571 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qk5lt\"" Oct 30 18:40:05.286734 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.286606 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Oct 30 18:40:05.291806 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.291782 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p"] Oct 30 18:40:05.303709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.303685 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.303824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.303717 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.303824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.303738 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f915fff9-985c-4353-b5fb-166e126440cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.303897 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.303873 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.303930 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.303912 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.303964 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.303929 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869lr\" (UniqueName: \"kubernetes.io/projected/f915fff9-985c-4353-b5fb-166e126440cb-kube-api-access-869lr\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.378051 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.378018 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Oct 30 18:40:05.405241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405206 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405246 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405267 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f915fff9-985c-4353-b5fb-166e126440cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405320 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405557 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405428 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405557 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405474 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-869lr\" (UniqueName: \"kubernetes.io/projected/f915fff9-985c-4353-b5fb-166e126440cb-kube-api-access-869lr\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405677 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405751 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405721 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.405821 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.405797 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.407646 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.407620 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f915fff9-985c-4353-b5fb-166e126440cb-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.407934 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.407918 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f915fff9-985c-4353-b5fb-166e126440cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.423924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.423900 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:05.432402 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.432381 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-869lr\" (UniqueName: \"kubernetes.io/projected/f915fff9-985c-4353-b5fb-166e126440cb-kube-api-access-869lr\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-cns7p\" (UID: \"f915fff9-985c-4353-b5fb-166e126440cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.571410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.571330 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:05.879841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:05.879805 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p"] Oct 30 18:40:05.880948 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:40:05.880918 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf915fff9_985c_4353_b5fb_166e126440cb.slice/crio-5aab255be0d2dc4e33e19dc2b0ded9df02644f6e29fb8cbe48d584cf85a87c86 WatchSource:0}: Error finding container 5aab255be0d2dc4e33e19dc2b0ded9df02644f6e29fb8cbe48d584cf85a87c86: Status 404 returned error can't find the container with id 5aab255be0d2dc4e33e19dc2b0ded9df02644f6e29fb8cbe48d584cf85a87c86 Oct 30 18:40:06.554194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:06.554131 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" event={"ID":"f915fff9-985c-4353-b5fb-166e126440cb","Type":"ContainerStarted","Data":"5aab255be0d2dc4e33e19dc2b0ded9df02644f6e29fb8cbe48d584cf85a87c86"} Oct 30 18:40:07.560790 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:07.560743 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" event={"ID":"f915fff9-985c-4353-b5fb-166e126440cb","Type":"ContainerStarted","Data":"091fe106105c95959fb99d6504ba6bed55da4f927fc0f7f56e0cf9244193e5c9"} Oct 30 18:40:07.572133 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:07.572112 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:07.608804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:07.608740 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" podStartSLOduration=1.5653395190000001 podStartE2EDuration="2.608727997s" totalImagesPullingTime="1.043388478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:07.607551535 +0000 UTC m=+524.722685470" watchObservedRunningTime="2026-10-30 18:40:07.608727997 +0000 UTC m=+524.723861914" Oct 30 18:40:09.741697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.741658 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg"] Oct 30 18:40:09.745416 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.745396 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.763951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.763918 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Oct 30 18:40:09.778275 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.778249 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg"] Oct 30 18:40:09.846715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.846685 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.846715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.846717 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.846909 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.846734 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.846909 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.846753 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk6k9\" (UniqueName: \"kubernetes.io/projected/58701ee9-4eb9-4637-a40e-b1611586455d-kube-api-access-mk6k9\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.846909 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.846901 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58701ee9-4eb9-4637-a40e-b1611586455d-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.847002 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.846942 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.864317 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.864287 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:09.948239 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948204 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948247 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948268 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948474 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948393 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk6k9\" (UniqueName: \"kubernetes.io/projected/58701ee9-4eb9-4637-a40e-b1611586455d-kube-api-access-mk6k9\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948515 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948491 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58701ee9-4eb9-4637-a40e-b1611586455d-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948575 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948558 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948667 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948649 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948727 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948691 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.948880 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.948861 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.950711 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.950690 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58701ee9-4eb9-4637-a40e-b1611586455d-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.950999 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.950978 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58701ee9-4eb9-4637-a40e-b1611586455d-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:09.973511 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:09.973489 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk6k9\" (UniqueName: \"kubernetes.io/projected/58701ee9-4eb9-4637-a40e-b1611586455d-kube-api-access-mk6k9\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-jffqg\" (UID: \"58701ee9-4eb9-4637-a40e-b1611586455d\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:10.058563 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:10.058477 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:10.358443 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:10.358418 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg"] Oct 30 18:40:10.359411 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:40:10.359381 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58701ee9_4eb9_4637_a40e_b1611586455d.slice/crio-7f4957891ed1656ac81e1157e31ebda287d4a50a4f1f2c5b4571bf8c0889daae WatchSource:0}: Error finding container 7f4957891ed1656ac81e1157e31ebda287d4a50a4f1f2c5b4571bf8c0889daae: Status 404 returned error can't find the container with id 7f4957891ed1656ac81e1157e31ebda287d4a50a4f1f2c5b4571bf8c0889daae Oct 30 18:40:10.574794 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:10.574765 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" event={"ID":"58701ee9-4eb9-4637-a40e-b1611586455d","Type":"ContainerStarted","Data":"7f4957891ed1656ac81e1157e31ebda287d4a50a4f1f2c5b4571bf8c0889daae"} Oct 30 18:40:11.580937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:11.580899 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" event={"ID":"58701ee9-4eb9-4637-a40e-b1611586455d","Type":"ContainerStarted","Data":"71782a5c4546f632ac97f2d144722418ec4fd99ad19e84c72dd6d892e4e2c57d"} Oct 30 18:40:11.628846 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:11.628796 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" podStartSLOduration=2.421378184 podStartE2EDuration="2.628782808s" totalImagesPullingTime="207.404624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:11.625978639 +0000 UTC m=+528.741112578" watchObservedRunningTime="2026-10-30 18:40:11.628782808 +0000 UTC m=+528.743916718" Oct 30 18:40:12.059627 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:12.059592 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:12.080483 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:12.080450 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:13.529262 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:13.529226 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:15.148310 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:15.148266 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:15.572583 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:15.572494 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:15.585284 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:15.585259 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:15.618218 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:15.618196 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-cns7p" Oct 30 18:40:15.884103 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:15.884068 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:16.083937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.083905 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:16.247023 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.246951 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:16.644845 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.644817 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd"] Oct 30 18:40:16.648392 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.648365 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.693626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.693598 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Oct 30 18:40:16.695575 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.695534 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd"] Oct 30 18:40:16.777601 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.777565 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:16.806359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.806336 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.806464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.806375 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.806464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.806411 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837eb667-3457-409a-9818-d502ec96f06e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.806567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.806528 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.806616 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.806581 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.806655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.806635 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsqx5\" (UniqueName: \"kubernetes.io/projected/837eb667-3457-409a-9818-d502ec96f06e-kube-api-access-jsqx5\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.907911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.907831 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.907911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.907889 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837eb667-3457-409a-9818-d502ec96f06e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.907926 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.907954 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.907985 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsqx5\" (UniqueName: \"kubernetes.io/projected/837eb667-3457-409a-9818-d502ec96f06e-kube-api-access-jsqx5\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.908012 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908396 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.908307 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908396 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.908318 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.908481 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.908399 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.910904 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.910883 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/837eb667-3457-409a-9818-d502ec96f06e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.911074 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.911057 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/837eb667-3457-409a-9818-d502ec96f06e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.931963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.931938 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsqx5\" (UniqueName: \"kubernetes.io/projected/837eb667-3457-409a-9818-d502ec96f06e-kube-api-access-jsqx5\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd\" (UID: \"837eb667-3457-409a-9818-d502ec96f06e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:16.961034 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:16.961007 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:17.269568 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:17.269543 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd"] Oct 30 18:40:17.270455 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:40:17.270422 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod837eb667_3457_409a_9818_d502ec96f06e.slice/crio-371ef2e56538e62ca0ee52bf0177e0aed3faeb56446823e81127f7edc0b9c06f WatchSource:0}: Error finding container 371ef2e56538e62ca0ee52bf0177e0aed3faeb56446823e81127f7edc0b9c06f: Status 404 returned error can't find the container with id 371ef2e56538e62ca0ee52bf0177e0aed3faeb56446823e81127f7edc0b9c06f Oct 30 18:40:17.620441 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:17.620410 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" event={"ID":"837eb667-3457-409a-9818-d502ec96f06e","Type":"ContainerStarted","Data":"25810af184712fe56011a17c840c1f746dea73948cecf2cde0ff1cdd72e12b05"} Oct 30 18:40:17.620578 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:17.620463 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" event={"ID":"837eb667-3457-409a-9818-d502ec96f06e","Type":"ContainerStarted","Data":"371ef2e56538e62ca0ee52bf0177e0aed3faeb56446823e81127f7edc0b9c06f"} Oct 30 18:40:17.664701 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:17.664655 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" podStartSLOduration=1.4498956060000001 podStartE2EDuration="1.664641475s" totalImagesPullingTime="214.745869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:17.663353719 +0000 UTC m=+534.778487634" watchObservedRunningTime="2026-10-30 18:40:17.664641475 +0000 UTC m=+534.779775392" Oct 30 18:40:17.962062 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:17.962032 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:18.305121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.305043 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4"] Oct 30 18:40:18.308874 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.308852 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.329192 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.329166 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4"] Oct 30 18:40:18.339666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.339643 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Oct 30 18:40:18.411443 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.411414 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:18.422000 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.421979 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.422072 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.422018 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqzgx\" (UniqueName: \"kubernetes.io/projected/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-kube-api-access-kqzgx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.422154 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.422125 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.422201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.422188 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.422236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.422228 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.422270 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.422244 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.522848 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.522809 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.522848 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.522849 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.522874 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.522972 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.523056 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523280 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.523101 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqzgx\" (UniqueName: \"kubernetes.io/projected/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-kube-api-access-kqzgx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523357 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.523325 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.523351 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.523422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.523373 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.525245 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.525224 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.525536 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.525519 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.572307 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.572228 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqzgx\" (UniqueName: \"kubernetes.io/projected/c1c41d57-c5a2-419c-8e72-67ae1cc874ea-kube-api-access-kqzgx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4\" (UID: \"c1c41d57-c5a2-419c-8e72-67ae1cc874ea\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.621078 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.621046 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:18.924293 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:18.924263 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4"] Oct 30 18:40:18.924981 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:40:18.924951 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1c41d57_c5a2_419c_8e72_67ae1cc874ea.slice/crio-ff1960355c325330541cd7c290628a81686a6c2a91def4c269b7c8660aea27c5 WatchSource:0}: Error finding container ff1960355c325330541cd7c290628a81686a6c2a91def4c269b7c8660aea27c5: Status 404 returned error can't find the container with id ff1960355c325330541cd7c290628a81686a6c2a91def4c269b7c8660aea27c5 Oct 30 18:40:19.634202 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:19.634166 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" event={"ID":"c1c41d57-c5a2-419c-8e72-67ae1cc874ea","Type":"ContainerStarted","Data":"758b905a99a1c53cac62a191a4d6581f77e65bc2fd17103d1970c6542e4cbfae"} Oct 30 18:40:19.634565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:19.634212 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" event={"ID":"c1c41d57-c5a2-419c-8e72-67ae1cc874ea","Type":"ContainerStarted","Data":"ff1960355c325330541cd7c290628a81686a6c2a91def4c269b7c8660aea27c5"} Oct 30 18:40:19.678575 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:19.678529 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" podStartSLOduration=1.447506247 podStartE2EDuration="1.67851476s" totalImagesPullingTime="231.008513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:19.676847585 +0000 UTC m=+536.791981500" watchObservedRunningTime="2026-10-30 18:40:19.67851476 +0000 UTC m=+536.793648676" Oct 30 18:40:20.059453 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:20.059375 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:20.071839 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:20.071810 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:20.188061 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:20.188026 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:20.621935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:20.621901 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:20.650613 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:20.650582 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-jffqg" Oct 30 18:40:26.962214 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:26.962178 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:26.974774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:26.974753 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:27.681853 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:27.681822 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd" Oct 30 18:40:28.621389 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:28.621356 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:28.633265 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:28.633240 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:28.686976 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:28.686943 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4" Oct 30 18:40:33.832026 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:40:33.831989 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:40:34.716382 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:34.716347 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:34.874090 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:34.874059 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:36.434699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:36.434664 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:36.602222 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:36.602192 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:38.089535 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.089495 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5"] Oct 30 18:40:38.093388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.093364 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.126968 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.126936 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd"] Oct 30 18:40:38.130996 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.130970 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.132819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.132782 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5"] Oct 30 18:40:38.145263 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.145244 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9bw4x\"" Oct 30 18:40:38.170119 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.170093 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd"] Oct 30 18:40:38.195284 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195255 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.195434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195290 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.195434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195307 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.195434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195324 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195385 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.195434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195422 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195464 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195498 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f32b9d16-e1d8-4d54-9434-261254241d83-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195524 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gm4m\" (UniqueName: \"kubernetes.io/projected/f32b9d16-e1d8-4d54-9434-261254241d83-kube-api-access-7gm4m\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195602 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.195787 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195707 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f32b9d16-e1d8-4d54-9434-261254241d83-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195787 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195746 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195852 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195802 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f32b9d16-e1d8-4d54-9434-261254241d83-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.195897 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195851 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xt8t\" (UniqueName: \"kubernetes.io/projected/72ebd198-f18a-4a53-a8d8-8ac880573147-kube-api-access-7xt8t\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.195957 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195932 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.196012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195969 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.196012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.195992 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72ebd198-f18a-4a53-a8d8-8ac880573147-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.196135 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.196116 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.297336 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297297 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f32b9d16-e1d8-4d54-9434-261254241d83-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.297336 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297337 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gm4m\" (UniqueName: \"kubernetes.io/projected/f32b9d16-e1d8-4d54-9434-261254241d83-kube-api-access-7gm4m\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297355 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297394 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f32b9d16-e1d8-4d54-9434-261254241d83-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297418 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297442 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f32b9d16-e1d8-4d54-9434-261254241d83-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297463 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xt8t\" (UniqueName: \"kubernetes.io/projected/72ebd198-f18a-4a53-a8d8-8ac880573147-kube-api-access-7xt8t\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297490 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297514 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297537 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72ebd198-f18a-4a53-a8d8-8ac880573147-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.297582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297579 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297649 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297689 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297715 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297750 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297776 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297794 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297829 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297861 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.297909 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298015 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298177 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f32b9d16-e1d8-4d54-9434-261254241d83-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298218 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298326 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72ebd198-f18a-4a53-a8d8-8ac880573147-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298498 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298519 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.298842 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298582 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.298842 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.298785 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.300161 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.300114 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f32b9d16-e1d8-4d54-9434-261254241d83-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.300275 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.300176 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.300345 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.300327 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f32b9d16-e1d8-4d54-9434-261254241d83-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.300419 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.300402 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.322352 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.322320 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f32b9d16-e1d8-4d54-9434-261254241d83-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.322651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.322630 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gm4m\" (UniqueName: \"kubernetes.io/projected/f32b9d16-e1d8-4d54-9434-261254241d83-kube-api-access-7gm4m\") pod \"maas-default-gateway-openshift-default-6dcf757899-qqbk5\" (UID: \"f32b9d16-e1d8-4d54-9434-261254241d83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.327339 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.327314 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72ebd198-f18a-4a53-a8d8-8ac880573147-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.327851 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.327831 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xt8t\" (UniqueName: \"kubernetes.io/projected/72ebd198-f18a-4a53-a8d8-8ac880573147-kube-api-access-7xt8t\") pod \"maas-default-gateway-openshift-default-6dcf757899-6njrd\" (UID: \"72ebd198-f18a-4a53-a8d8-8ac880573147\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.406595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.406562 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:38.443994 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.443963 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:38.724412 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.724383 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5"] Oct 30 18:40:38.724890 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:40:38.724867 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32b9d16_e1d8_4d54_9434_261254241d83.slice/crio-1db533d141fb5333d202357256cec86c14799807c8c24a623db54c9ddf50eb03 WatchSource:0}: Error finding container 1db533d141fb5333d202357256cec86c14799807c8c24a623db54c9ddf50eb03: Status 404 returned error can't find the container with id 1db533d141fb5333d202357256cec86c14799807c8c24a623db54c9ddf50eb03 Oct 30 18:40:38.726946 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.726911 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:38.727021 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.726973 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:38.727021 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.726998 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:38.768166 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.768132 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd"] Oct 30 18:40:38.769109 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:40:38.769084 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ebd198_f18a_4a53_a8d8_8ac880573147.slice/crio-6f450cdc5dafa470a3babcfa3a9b11503733acb8072eedaaa292168edd84d03b WatchSource:0}: Error finding container 6f450cdc5dafa470a3babcfa3a9b11503733acb8072eedaaa292168edd84d03b: Status 404 returned error can't find the container with id 6f450cdc5dafa470a3babcfa3a9b11503733acb8072eedaaa292168edd84d03b Oct 30 18:40:38.771694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.771661 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:38.771772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.771726 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:38.771772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:38.771753 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:40:39.766972 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:39.766937 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" event={"ID":"72ebd198-f18a-4a53-a8d8-8ac880573147","Type":"ContainerStarted","Data":"a201fbbdbc3ae3866940476cc1d1690577c953aab99106bbcc81cfeae41d29f7"} Oct 30 18:40:39.767591 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:39.767529 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" event={"ID":"72ebd198-f18a-4a53-a8d8-8ac880573147","Type":"ContainerStarted","Data":"6f450cdc5dafa470a3babcfa3a9b11503733acb8072eedaaa292168edd84d03b"} Oct 30 18:40:39.769013 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:39.768983 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" event={"ID":"f32b9d16-e1d8-4d54-9434-261254241d83","Type":"ContainerStarted","Data":"550d1e651e9174d0c51e83570406a52ab0534270a0d651bc912a017e2af1fc59"} Oct 30 18:40:39.769115 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:39.769035 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" event={"ID":"f32b9d16-e1d8-4d54-9434-261254241d83","Type":"ContainerStarted","Data":"1db533d141fb5333d202357256cec86c14799807c8c24a623db54c9ddf50eb03"} Oct 30 18:40:39.833172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:39.831346 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podStartSLOduration=1.8313278670000002 podStartE2EDuration="1.831327867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:39.830602879 +0000 UTC m=+556.945736816" watchObservedRunningTime="2026-10-30 18:40:39.831327867 +0000 UTC m=+556.946461783" Oct 30 18:40:39.904690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:39.904629 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podStartSLOduration=1.904615416 podStartE2EDuration="1.904615416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:39.903365064 +0000 UTC m=+557.018498999" watchObservedRunningTime="2026-10-30 18:40:39.904615416 +0000 UTC m=+557.019749406" Oct 30 18:40:40.407322 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:40.407281 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:40.408626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:40.408600 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:40:40.408730 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:40.408651 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Oct 30 18:40:40.448171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:40.447968 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:40.448171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:40.448156 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:40:40.448399 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:40.448205 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Oct 30 18:40:41.407130 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:41.407093 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:40:41.407639 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:41.407177 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Oct 30 18:40:41.456170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:41.449078 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:40:41.456170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:41.449137 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Oct 30 18:40:42.407857 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:42.407819 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:40:42.408289 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:42.407872 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Oct 30 18:40:42.449169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:42.445438 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:40:42.449169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:42.445500 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Oct 30 18:40:43.407365 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:43.407338 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:40:43.409551 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:43.407511 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Oct 30 18:40:43.445560 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:43.445523 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:40:43.445816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:43.445787 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Oct 30 18:40:44.407944 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:44.407904 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:40:44.408172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:44.407971 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Oct 30 18:40:44.449169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:44.445308 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:40:44.449169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:44.445369 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Oct 30 18:40:45.407254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:45.407203 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Oct 30 18:40:45.407464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:45.407279 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Oct 30 18:40:45.445225 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:45.445180 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Oct 30 18:40:45.445414 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:45.445245 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Oct 30 18:40:46.888818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:46.888783 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:46.889993 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:46.889965 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:46.890774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:46.890755 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" Oct 30 18:40:46.941354 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:46.941333 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:46.941650 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:46.941630 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:40:46.942439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:40:46.942418 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" Oct 30 18:41:03.833180 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:03.833128 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:41:05.869761 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:05.869732 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:06.063030 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.062998 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:06.239459 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.239369 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:06.474655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.474603 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:06.488164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.488123 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g"] Oct 30 18:41:06.493262 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.493185 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.509466 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.509443 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f7d8050f-openshift-default-dockercfg-h2c9k\"" Oct 30 18:41:06.523447 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.523420 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g"] Oct 30 18:41:06.615390 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615351 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.615578 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615470 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.615578 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615508 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.615578 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615551 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.616254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615665 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.616254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615756 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.616254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.615786 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.616254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.616214 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.616495 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.616288 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mg2d\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.623894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.623871 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9"] Oct 30 18:41:06.628068 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.628045 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.653450 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.653423 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-48afdd9e-openshift-default-dockercfg-z87sl\"" Oct 30 18:41:06.665747 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.665715 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9"] Oct 30 18:41:06.675866 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.675827 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:06.717553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717523 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.717553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717555 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.717811 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717582 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.717811 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717615 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.717811 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717744 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mg2d\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.717963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717812 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.717963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717860 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717970 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.717968 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ntmq\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718029 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718073 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718072 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718107 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718192 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718222 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718258 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718286 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718324 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718366 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718401 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.718558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718551 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.718832 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.718810 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.719086 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.719066 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.719971 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.719946 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.720657 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.720636 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.745373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.743194 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.745373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.745000 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mg2d\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d\") pod \"e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.808770 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.808734 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:06.819948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.819920 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.819989 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ntmq\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820017 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820034 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820055 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820278 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820086 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820278 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820119 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820278 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820138 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820278 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820208 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820597 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820569 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.820702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.820606 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.821040 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.821018 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.821206 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.821186 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.821269 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.821207 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.822900 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.822877 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.823217 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.823196 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.841224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.841181 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:06.844390 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.844356 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.844512 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.844487 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ntmq\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq\") pod \"e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:06.942775 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:06.942740 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:07.000990 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.000896 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59"] Oct 30 18:41:07.009517 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.009494 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.015074 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.014832 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:07.032012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.029821 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-b77c9bba-gw-openshift-default-dockercfg-7pxq9\"" Oct 30 18:41:07.045691 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.045485 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59"] Oct 30 18:41:07.123510 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.123468 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.123682 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.123527 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.123682 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.123564 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.123682 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.123642 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lb5m\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.123799 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.123714 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.123930 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.123910 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.124107 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.124040 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.124107 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.124084 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.124338 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.124177 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.125997 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.125971 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g"] Oct 30 18:41:07.127437 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:07.127413 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7343159_7895_4eb8_8a5b_57d1c28eecef.slice/crio-607ded75f00c330cb6933233b6d25b2b1b1d2a5c73920824e408809cb24c2b49 WatchSource:0}: Error finding container 607ded75f00c330cb6933233b6d25b2b1b1d2a5c73920824e408809cb24c2b49: Status 404 returned error can't find the container with id 607ded75f00c330cb6933233b6d25b2b1b1d2a5c73920824e408809cb24c2b49 Oct 30 18:41:07.130378 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.130342 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.130485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.130424 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.130485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.130472 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.205047 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.205018 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:07.224994 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.224966 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225139 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225002 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225139 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225023 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225139 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225115 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225139 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225159 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225387 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225192 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225387 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225231 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lb5m\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225387 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225256 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225387 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225290 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225590 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225549 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225691 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225667 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.225838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.225815 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.226171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.226125 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.226739 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.226715 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.227697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.227676 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.228696 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.228668 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.249353 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.249320 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.249603 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.249578 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lb5m\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m\") pod \"e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.258837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.258814 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9"] Oct 30 18:41:07.259577 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:07.259550 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf6f3142_17d0_4d3e_ad3c_13a65477befb.slice/crio-9ca1b42419db47dba528d0d3a9deab2997c0a82cd9b31584b14d97735424adb7 WatchSource:0}: Error finding container 9ca1b42419db47dba528d0d3a9deab2997c0a82cd9b31584b14d97735424adb7: Status 404 returned error can't find the container with id 9ca1b42419db47dba528d0d3a9deab2997c0a82cd9b31584b14d97735424adb7 Oct 30 18:41:07.261916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.261886 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.261984 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.261947 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.261984 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.261976 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.339058 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.339028 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:07.646566 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.646539 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59"] Oct 30 18:41:07.647799 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:07.647766 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f70766_4490_4d10_a6dd_aaed2ad4cead.slice/crio-8d17b0b7d07a7ee51d9b6868bbb0f41b6244e9fa677a819f612f241d7de20845 WatchSource:0}: Error finding container 8d17b0b7d07a7ee51d9b6868bbb0f41b6244e9fa677a819f612f241d7de20845: Status 404 returned error can't find the container with id 8d17b0b7d07a7ee51d9b6868bbb0f41b6244e9fa677a819f612f241d7de20845 Oct 30 18:41:07.650220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.650186 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.650330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.650267 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.650330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.650311 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:07.917313 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.917218 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" event={"ID":"bf6f3142-17d0-4d3e-ad3c-13a65477befb","Type":"ContainerStarted","Data":"a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995"} Oct 30 18:41:07.917313 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.917290 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" event={"ID":"bf6f3142-17d0-4d3e-ad3c-13a65477befb","Type":"ContainerStarted","Data":"9ca1b42419db47dba528d0d3a9deab2997c0a82cd9b31584b14d97735424adb7"} Oct 30 18:41:07.918848 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.918815 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" event={"ID":"a7343159-7895-4eb8-8a5b-57d1c28eecef","Type":"ContainerStarted","Data":"e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70"} Oct 30 18:41:07.919004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.918865 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" event={"ID":"a7343159-7895-4eb8-8a5b-57d1c28eecef","Type":"ContainerStarted","Data":"607ded75f00c330cb6933233b6d25b2b1b1d2a5c73920824e408809cb24c2b49"} Oct 30 18:41:07.920409 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.920385 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" event={"ID":"62f70766-4490-4d10-a6dd-aaed2ad4cead","Type":"ContainerStarted","Data":"c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4"} Oct 30 18:41:07.920532 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.920428 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" event={"ID":"62f70766-4490-4d10-a6dd-aaed2ad4cead","Type":"ContainerStarted","Data":"8d17b0b7d07a7ee51d9b6868bbb0f41b6244e9fa677a819f612f241d7de20845"} Oct 30 18:41:07.943641 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.943612 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:07.957251 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.957214 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:07.964037 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:07.963990 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podStartSLOduration=1.9639777889999999 podStartE2EDuration="1.963977789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:07.963569333 +0000 UTC m=+585.078703261" watchObservedRunningTime="2026-10-30 18:41:07.963977789 +0000 UTC m=+585.079111707" Oct 30 18:41:08.013648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.013589 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podStartSLOduration=2.013574479 podStartE2EDuration="2.013574479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:08.012095052 +0000 UTC m=+585.127228988" watchObservedRunningTime="2026-10-30 18:41:08.013574479 +0000 UTC m=+585.128708396" Oct 30 18:41:08.058392 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.058343 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podStartSLOduration=2.058326853 podStartE2EDuration="2.058326853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:08.05784475 +0000 UTC m=+585.172978667" watchObservedRunningTime="2026-10-30 18:41:08.058326853 +0000 UTC m=+585.173460770" Oct 30 18:41:08.339590 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.339489 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:08.346135 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.346105 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:08.358991 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.358961 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t"] Oct 30 18:41:08.363449 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.363425 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.381190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.381166 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ed302a08-openshift-default-dockercfg-5v4xl\"" Oct 30 18:41:08.387506 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.387480 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t"] Oct 30 18:41:08.444732 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.444666 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:08.539423 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539388 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539615 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539441 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539615 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539589 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539635 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539735 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539717 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f9t9\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539771 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539762 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539806 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539785 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539875 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539863 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.539910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.539882 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.605229 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.605128 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:08.640970 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.640936 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641197 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.640978 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641197 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641115 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641294 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641225 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641345 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641315 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641416 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641382 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641487 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641471 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f9t9\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641661 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641635 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641725 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641647 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641765 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641734 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641827 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641803 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641873 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641859 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.641916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.641804 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.642037 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.642017 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.643611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.643587 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.644254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.644236 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.665719 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.665679 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f9t9\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.665808 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.665752 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token\") pod \"e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.677594 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.677568 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:08.809531 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.809498 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:08.814500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.814469 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:08.925888 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.925855 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:08.926057 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.925912 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:08.926460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.926437 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:08.926572 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.926489 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:08.948836 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.948812 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:08.997633 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:08.997601 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t"] Oct 30 18:41:09.003947 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.003450 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:09.003947 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.003552 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:09.003947 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.003598 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:09.778776 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.778737 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:09.932375 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.932327 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" event={"ID":"c034bd64-3f9e-4b7a-a2cd-607b853adaae","Type":"ContainerStarted","Data":"38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670"} Oct 30 18:41:09.932614 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.932463 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" event={"ID":"c034bd64-3f9e-4b7a-a2cd-607b853adaae","Type":"ContainerStarted","Data":"3a5168a337c0ca4c28f8414ce840a092487b03cf18aa91bd40ea6825c12cc14c"} Oct 30 18:41:09.932888 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.932869 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:09.933778 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.933760 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:09.991602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:09.991569 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:10.006348 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.006295 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podStartSLOduration=2.006283004 podStartE2EDuration="2.006283004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:09.993458361 +0000 UTC m=+587.108592297" watchObservedRunningTime="2026-10-30 18:41:10.006283004 +0000 UTC m=+587.121416921" Oct 30 18:41:10.210190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.210155 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:10.369311 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.369279 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:10.678281 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.678250 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:10.684358 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.684332 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:10.936993 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.936902 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:10.937944 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:10.937922 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:11.261113 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:11.261033 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:11.413438 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:11.413407 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:11.576587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:11.576512 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:12.885782 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:12.885750 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:13.088720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:13.088688 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:13.355337 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:13.355257 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:13.760542 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:13.760511 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:13.935714 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:13.935675 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:14.097913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:14.097829 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:14.260710 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:14.260682 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:14.668198 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:14.665046 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:14.817304 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:14.817263 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:15.025201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:15.025092 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:15.174214 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:15.174181 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:16.225052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:16.225015 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:16.390256 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:16.390220 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:16.796964 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:16.796873 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:18.004650 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:18.004618 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:18.205454 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:18.205423 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:18.612816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:18.612778 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:18.777901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:18.777860 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:18.940483 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:18.940450 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:20.280690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:20.280660 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:20.490729 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:20.490694 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:20.662390 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:20.662358 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:20.812577 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:20.812546 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:22.590207 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:22.590171 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:23.098303 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:23.098271 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:41:23.100705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:23.100682 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:41:23.102980 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:23.102960 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:41:23.105245 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:23.105227 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:41:24.353021 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:24.352984 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:24.985302 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:24.985261 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:41:24.985487 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:24.985325 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:41:26.045953 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:26.045851 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:26.239170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:26.230519 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:27.890674 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:27.890636 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:28.108929 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:28.108900 5070 generic.go:356] "Generic (PLEG): container finished" podID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerID="c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4" exitCode=0 Oct 30 18:41:28.109110 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:28.108975 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" event={"ID":"62f70766-4490-4d10-a6dd-aaed2ad4cead","Type":"ContainerDied","Data":"c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4"} Oct 30 18:41:28.109433 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:28.109415 5070 scope.go:122] "RemoveContainer" containerID="c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4" Oct 30 18:41:28.110667 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:28.110639 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:28.110746 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:28.110702 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:28.110746 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:28.110735 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:29.137179 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:29.132745 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" event={"ID":"62f70766-4490-4d10-a6dd-aaed2ad4cead","Type":"ContainerStarted","Data":"b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661"} Oct 30 18:41:29.341005 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:29.340555 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:29.341005 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:29.340684 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 30 18:41:29.341005 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:29.340729 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 30 18:41:29.437058 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:29.436961 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:30.343170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:30.342522 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 30 18:41:30.343170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:30.342589 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 30 18:41:31.340466 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.335225 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8"] Oct 30 18:41:31.341720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.341263 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 30 18:41:31.341720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.341331 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 30 18:41:31.344730 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.344706 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.374177 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.350914 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8"] Oct 30 18:41:31.400169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.363776 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:41:31.400169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.364753 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:41:31.400169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.365727 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:31.412363 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.412330 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.412677 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.412653 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.412892 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.412873 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxssn\" (UniqueName: \"kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.550184 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.521712 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.550184 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.521827 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.550184 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.521931 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxssn\" (UniqueName: \"kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.550184 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.523111 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.550184 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.548454 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.592932 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.582846 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxssn\" (UniqueName: \"kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn\") pod \"maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:31.712583 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:31.712545 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:32.140418 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:32.140385 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8"] Oct 30 18:41:32.148252 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:32.148219 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb56b2f_0472_40c9_ba73_bbc5bbfe13cd.slice/crio-d504a2c4ce56d2f7a4a518f4d4243d2240f20a5d65c86736572383542e973a14 WatchSource:0}: Error finding container d504a2c4ce56d2f7a4a518f4d4243d2240f20a5d65c86736572383542e973a14: Status 404 returned error can't find the container with id d504a2c4ce56d2f7a4a518f4d4243d2240f20a5d65c86736572383542e973a14 Oct 30 18:41:32.151678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:32.150865 5070 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:41:32.201922 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:32.201880 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" event={"ID":"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd","Type":"ContainerStarted","Data":"d504a2c4ce56d2f7a4a518f4d4243d2240f20a5d65c86736572383542e973a14"} Oct 30 18:41:32.341171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:32.340036 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 30 18:41:32.341171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:32.340106 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 30 18:41:33.291929 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:33.291864 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:33.339847 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:33.339809 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 30 18:41:33.340109 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:33.340076 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 30 18:41:33.871620 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:33.871533 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:41:34.339512 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:34.339419 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 30 18:41:34.339512 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:34.339490 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 30 18:41:35.184623 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:35.184497 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:35.344521 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:35.344488 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:35.344999 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:35.344907 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:35.345689 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:35.345666 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:36.636586 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:36.254462 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" event={"ID":"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd","Type":"ContainerStarted","Data":"bc0344b25360be56704a85c4a79f71dcd9a3b56a946c9e30db911f2370621f92"} Oct 30 18:41:36.636586 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:36.303764 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" podStartSLOduration=1.631680784 podStartE2EDuration="5.303751003s" totalImagesPullingTime="3.672070219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:36.303511817 +0000 UTC m=+613.418645734" watchObservedRunningTime="2026-10-30 18:41:36.303751003 +0000 UTC m=+613.418884920" Oct 30 18:41:36.987221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:36.982961 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:37.261974 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:37.261885 5070 generic.go:356] "Generic (PLEG): container finished" podID="feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" containerID="bc0344b25360be56704a85c4a79f71dcd9a3b56a946c9e30db911f2370621f92" exitCode=0 Oct 30 18:41:37.262170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:37.261971 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" event={"ID":"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd","Type":"ContainerDied","Data":"bc0344b25360be56704a85c4a79f71dcd9a3b56a946c9e30db911f2370621f92"} Oct 30 18:41:38.268846 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.268814 5070 generic.go:356] "Generic (PLEG): container finished" podID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerID="38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670" exitCode=0 Oct 30 18:41:38.269380 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.268892 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" event={"ID":"c034bd64-3f9e-4b7a-a2cd-607b853adaae","Type":"ContainerDied","Data":"38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670"} Oct 30 18:41:38.269380 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.269348 5070 scope.go:122] "RemoveContainer" containerID="38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670" Oct 30 18:41:38.270568 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.270533 5070 generic.go:356] "Generic (PLEG): container finished" podID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerID="a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995" exitCode=0 Oct 30 18:41:38.270677 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.270567 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" event={"ID":"bf6f3142-17d0-4d3e-ad3c-13a65477befb","Type":"ContainerDied","Data":"a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995"} Oct 30 18:41:38.270824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.270791 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.270928 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.270867 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.270928 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.270916 5070 scope.go:122] "RemoveContainer" containerID="a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995" Oct 30 18:41:38.271037 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.270916 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.272332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.272303 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.272448 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.272380 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.272448 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.272431 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.272654 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.272634 5070 generic.go:356] "Generic (PLEG): container finished" podID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerID="e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70" exitCode=0 Oct 30 18:41:38.272730 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.272662 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" event={"ID":"a7343159-7895-4eb8-8a5b-57d1c28eecef","Type":"ContainerDied","Data":"e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70"} Oct 30 18:41:38.273156 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.273130 5070 scope.go:122] "RemoveContainer" containerID="e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70" Oct 30 18:41:38.274297 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.274271 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.274415 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.274343 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.274415 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.274395 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:38.520176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.517971 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:38.632908 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.632873 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca\") pod \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " Oct 30 18:41:38.633099 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.632956 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca\") pod \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " Oct 30 18:41:38.633099 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.633021 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn\" (UniqueName: \"kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn\") pod \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\" (UID: \"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd\") " Oct 30 18:41:38.633437 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.633405 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca" pod "feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" (UID: "feb56b2f-0472-40c9-ba73-bbc5bbfe13cd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:38.635373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.635342 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn" pod "feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" (UID: "feb56b2f-0472-40c9-ba73-bbc5bbfe13cd"). InnerVolumeSpecName "kube-api-access-dxssn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:38.635473 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.635348 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca" pod "feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" (UID: "feb56b2f-0472-40c9-ba73-bbc5bbfe13cd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:38.735034 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.734998 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-xks-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:38.735201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.735039 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-maas-api-openshift-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:38.735201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.735055 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxssn\" (UniqueName: \"kubernetes.io/projected/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd-kube-api-access-dxssn\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:38.842103 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.841936 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:38.926345 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:38.926252 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:39.295419 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.295388 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" event={"ID":"c034bd64-3f9e-4b7a-a2cd-607b853adaae","Type":"ContainerStarted","Data":"62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b"} Oct 30 18:41:39.298709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.298675 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" event={"ID":"bf6f3142-17d0-4d3e-ad3c-13a65477befb","Type":"ContainerStarted","Data":"298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d"} Oct 30 18:41:39.301568 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.301542 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" event={"ID":"a7343159-7895-4eb8-8a5b-57d1c28eecef","Type":"ContainerStarted","Data":"a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431"} Oct 30 18:41:39.304501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.303585 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" event={"ID":"feb56b2f-0472-40c9-ba73-bbc5bbfe13cd","Type":"ContainerDied","Data":"d504a2c4ce56d2f7a4a518f4d4243d2240f20a5d65c86736572383542e973a14"} Oct 30 18:41:39.304501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.303627 5070 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d504a2c4ce56d2f7a4a518f4d4243d2240f20a5d65c86736572383542e973a14" Oct 30 18:41:39.304501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.303680 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8" Oct 30 18:41:39.678169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.677742 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:39.683348 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.683312 5070 patch_prober.go:29] interesting pod/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 18:41:39.683855 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.683822 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 18:41:39.809703 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.809673 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:39.810592 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.810569 5070 patch_prober.go:29] interesting pod/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:41:39.810660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.810619 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 18:41:39.933463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.933314 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:39.944360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.944315 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:39.946283 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.946256 5070 patch_prober.go:29] interesting pod/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 18:41:39.946517 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:39.946469 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 18:41:40.474170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.473677 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:40.710169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.683802 5070 patch_prober.go:29] interesting pod/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 18:41:40.713173 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.710472 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 18:41:40.749226 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.741779 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:40.810109 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.810060 5070 patch_prober.go:29] interesting pod/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:41:40.811179 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.810816 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 18:41:40.911381 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.911306 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:40.946625 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.945260 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:40.969181 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.966308 5070 patch_prober.go:29] interesting pod/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 18:41:40.969181 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:40.966386 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 18:41:41.683172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:41.680330 5070 patch_prober.go:29] interesting pod/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 18:41:41.683172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:41.680412 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 18:41:41.809708 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:41.809666 5070 patch_prober.go:29] interesting pod/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:41:41.809884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:41.809724 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 18:41:41.948690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:41.948599 5070 patch_prober.go:29] interesting pod/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 18:41:41.948951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:41.948922 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 18:41:42.009189 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.009140 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:42.172803 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.172728 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:42.365221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.364212 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:42.681170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.680556 5070 patch_prober.go:29] interesting pod/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 18:41:42.681170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.680619 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 18:41:42.809223 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.809187 5070 patch_prober.go:29] interesting pod/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:41:42.809636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.809240 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 18:41:42.946763 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.946670 5070 patch_prober.go:29] interesting pod/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 18:41:42.947019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:42.946992 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 18:41:43.693376 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:43.692794 5070 patch_prober.go:29] interesting pod/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 30 18:41:43.693376 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:43.692871 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 30 18:41:43.809267 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:43.809211 5070 patch_prober.go:29] interesting pod/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Oct 30 18:41:43.809667 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:43.809292 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Oct 30 18:41:43.951394 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:43.951292 5070 patch_prober.go:29] interesting pod/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Oct 30 18:41:43.951681 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:43.951633 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Oct 30 18:41:44.089877 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.089841 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:44.764155 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.764112 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:44.765656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.765624 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:44.919731 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.919661 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:44.920661 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.920638 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:44.921659 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.921640 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:44.947137 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.947114 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:44.948086 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:44.948066 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:45.615281 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:45.615248 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:46.102343 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.102284 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:46.192554 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.192516 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8"] Oct 30 18:41:46.206846 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.206810 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7d8050f-kkbc8"] Oct 30 18:41:46.292584 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.292549 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:46.522921 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.522887 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:46.773008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.772926 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:46.932214 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:46.932177 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:47.032185 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:47.032096 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" path="/var/lib/kubelet/pods/feb56b2f-0472-40c9-ba73-bbc5bbfe13cd/volumes" Oct 30 18:41:47.921887 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:47.921855 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:48.026560 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.026528 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g"] Oct 30 18:41:48.026766 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.026707 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" containerID="cri-o://a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431" gracePeriod=30 Oct 30 18:41:48.094257 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.094224 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:48.327976 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.327903 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:48.504372 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.504339 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:48.651595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.651543 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:48.714772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.714733 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59"] Oct 30 18:41:48.715838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.715794 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" containerID="cri-o://b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661" gracePeriod=30 Oct 30 18:41:48.803038 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.802987 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:48.958532 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:48.958429 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:49.117472 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.117439 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:49.119374 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.119349 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9"] Oct 30 18:41:49.119586 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.119561 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" containerID="cri-o://298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d" gracePeriod=30 Oct 30 18:41:49.272843 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.272744 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:49.503488 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.503442 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:49.675824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.675790 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:49.825243 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:49.825209 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:50.097211 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.097114 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:50.260634 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.260593 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:50.347189 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.347152 5070 patch_prober.go:29] interesting pod/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:41:50.347377 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.347209 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:41:50.468363 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.468326 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:50.785772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.785736 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:50.801272 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.801245 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t"] Oct 30 18:41:50.801546 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.801505 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" containerID="cri-o://62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b" gracePeriod=30 Oct 30 18:41:50.958976 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:50.958936 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:51.548626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:51.548597 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:51.768083 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:51.768027 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:52.292602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:52.292570 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:52.443423 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:52.443383 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:52.953574 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:52.953538 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:53.245314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.245239 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:53.285411 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.285384 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:53.311371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311344 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311531 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311398 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311531 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311445 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311531 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311495 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311558 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311623 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311660 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311690 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311914 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311731 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs\") pod \"a7343159-7895-4eb8-8a5b-57d1c28eecef\" (UID: \"a7343159-7895-4eb8-8a5b-57d1c28eecef\") " Oct 30 18:41:53.311914 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311764 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:53.312019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.311989 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:53.312019 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312002 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:53.312196 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312090 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:53.312255 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312233 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:53.312562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312539 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.312648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312567 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.312648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312581 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.312648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312596 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7343159-7895-4eb8-8a5b-57d1c28eecef-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.312648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.312609 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.313962 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.313938 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:53.314047 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.314021 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:53.314271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.314213 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:41:53.320587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.320566 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d" pod "a7343159-7895-4eb8-8a5b-57d1c28eecef" (UID: "a7343159-7895-4eb8-8a5b-57d1c28eecef"). InnerVolumeSpecName "kube-api-access-5mg2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:53.414190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.414123 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.414190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.414165 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.414190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.414181 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7343159-7895-4eb8-8a5b-57d1c28eecef-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.414190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.414193 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mg2d\" (UniqueName: \"kubernetes.io/projected/a7343159-7895-4eb8-8a5b-57d1c28eecef-kube-api-access-5mg2d\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:53.454849 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.454819 5070 generic.go:356] "Generic (PLEG): container finished" podID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerID="a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431" exitCode=0 Oct 30 18:41:53.455001 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.454867 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" event={"ID":"a7343159-7895-4eb8-8a5b-57d1c28eecef","Type":"ContainerDied","Data":"a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431"} Oct 30 18:41:53.455001 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.454870 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" Oct 30 18:41:53.455001 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.454907 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g" event={"ID":"a7343159-7895-4eb8-8a5b-57d1c28eecef","Type":"ContainerDied","Data":"607ded75f00c330cb6933233b6d25b2b1b1d2a5c73920824e408809cb24c2b49"} Oct 30 18:41:53.455001 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.454932 5070 scope.go:122] "RemoveContainer" containerID="a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431" Oct 30 18:41:53.464284 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.464268 5070 scope.go:122] "RemoveContainer" containerID="e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70" Oct 30 18:41:53.472669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.472645 5070 scope.go:122] "RemoveContainer" containerID="a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431" Oct 30 18:41:53.472909 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:53.472890 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431\": container with ID starting with a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431 not found: ID does not exist" containerID="a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431" Oct 30 18:41:53.472989 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.472924 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431"} err="failed to get container status \"a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431\": rpc error: code = NotFound desc = could not find container \"a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431\": container with ID starting with a7f541d1476d6f9c76136ae067bf0837252fc9b565e681cc61dcdf072c077431 not found: ID does not exist" Oct 30 18:41:53.472989 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.472959 5070 scope.go:122] "RemoveContainer" containerID="e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70" Oct 30 18:41:53.473226 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:53.473208 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70\": container with ID starting with e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70 not found: ID does not exist" containerID="e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70" Oct 30 18:41:53.473269 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.473233 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70"} err="failed to get container status \"e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70\": rpc error: code = NotFound desc = could not find container \"e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70\": container with ID starting with e00c5f6fc1cc62b19a17e4a5a652f6752664b95885231219f7487637a9f54b70 not found: ID does not exist" Oct 30 18:41:53.517534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.517461 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g"] Oct 30 18:41:53.520392 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.520365 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f7d8050f-openshift-default-7cc75b8cb9-56t2g"] Oct 30 18:41:53.544134 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.544102 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:53.721010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.720971 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:53.948945 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.948909 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:53.977617 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:53.977596 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:54.023464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023442 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023511 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023546 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023604 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023632 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023692 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023788 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023833 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.023891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023780 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.023891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023861 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket\") pod \"62f70766-4490-4d10-a6dd-aaed2ad4cead\" (UID: \"62f70766-4490-4d10-a6dd-aaed2ad4cead\") " Oct 30 18:41:54.024103 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.023886 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.024184 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024111 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.024236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024208 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.024287 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024234 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:54.024363 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024344 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.024429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024366 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.024429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024378 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.024429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024395 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f70766-4490-4d10-a6dd-aaed2ad4cead-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.024429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.024409 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.025669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.025647 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.025779 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.025763 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:41:54.026141 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.026118 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:54.026211 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.026125 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m" pod "62f70766-4490-4d10-a6dd-aaed2ad4cead" (UID: "62f70766-4490-4d10-a6dd-aaed2ad4cead"). InnerVolumeSpecName "kube-api-access-8lb5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:54.125073 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.125044 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.125073 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.125072 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.125292 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.125086 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lb5m\" (UniqueName: \"kubernetes.io/projected/62f70766-4490-4d10-a6dd-aaed2ad4cead-kube-api-access-8lb5m\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.125292 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.125099 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f70766-4490-4d10-a6dd-aaed2ad4cead-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.388439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.388377 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:54.428445 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428417 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428462 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428520 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428554 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428593 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428615 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428673 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428794 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.428948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428823 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.428948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428833 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.428948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428834 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:54.428948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.428870 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs\") pod \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\" (UID: \"bf6f3142-17d0-4d3e-ad3c-13a65477befb\") " Oct 30 18:41:54.429208 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.429023 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.429268 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.429199 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.429268 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.429255 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.429386 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.429275 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.429386 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.429288 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.429386 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.429302 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.430910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.430890 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:54.431011 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.430921 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:41:54.431329 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.431307 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:54.431329 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.431315 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq" pod "bf6f3142-17d0-4d3e-ad3c-13a65477befb" (UID: "bf6f3142-17d0-4d3e-ad3c-13a65477befb"). InnerVolumeSpecName "kube-api-access-4ntmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:54.463749 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.463724 5070 generic.go:356] "Generic (PLEG): container finished" podID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerID="298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d" exitCode=0 Oct 30 18:41:54.463860 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.463778 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" Oct 30 18:41:54.463860 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.463797 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" event={"ID":"bf6f3142-17d0-4d3e-ad3c-13a65477befb","Type":"ContainerDied","Data":"298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d"} Oct 30 18:41:54.463975 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.463866 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9" event={"ID":"bf6f3142-17d0-4d3e-ad3c-13a65477befb","Type":"ContainerDied","Data":"9ca1b42419db47dba528d0d3a9deab2997c0a82cd9b31584b14d97735424adb7"} Oct 30 18:41:54.463975 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.463900 5070 scope.go:122] "RemoveContainer" containerID="298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d" Oct 30 18:41:54.466775 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.466750 5070 generic.go:356] "Generic (PLEG): container finished" podID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerID="b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661" exitCode=0 Oct 30 18:41:54.466868 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.466809 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" Oct 30 18:41:54.466868 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.466823 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" event={"ID":"62f70766-4490-4d10-a6dd-aaed2ad4cead","Type":"ContainerDied","Data":"b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661"} Oct 30 18:41:54.466968 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.466882 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59" event={"ID":"62f70766-4490-4d10-a6dd-aaed2ad4cead","Type":"ContainerDied","Data":"8d17b0b7d07a7ee51d9b6868bbb0f41b6244e9fa677a819f612f241d7de20845"} Oct 30 18:41:54.474059 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.474041 5070 scope.go:122] "RemoveContainer" containerID="a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995" Oct 30 18:41:54.483746 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.483730 5070 scope.go:122] "RemoveContainer" containerID="298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d" Oct 30 18:41:54.484050 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:54.484029 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d\": container with ID starting with 298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d not found: ID does not exist" containerID="298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d" Oct 30 18:41:54.484100 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.484066 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d"} err="failed to get container status \"298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d\": rpc error: code = NotFound desc = could not find container \"298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d\": container with ID starting with 298333cfdd9fa72bc70f2275f42d4d8d1af6e45856ad9aae50eeeb7e324c487d not found: ID does not exist" Oct 30 18:41:54.484100 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.484083 5070 scope.go:122] "RemoveContainer" containerID="a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995" Oct 30 18:41:54.484330 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:54.484311 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995\": container with ID starting with a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995 not found: ID does not exist" containerID="a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995" Oct 30 18:41:54.484398 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.484339 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995"} err="failed to get container status \"a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995\": rpc error: code = NotFound desc = could not find container \"a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995\": container with ID starting with a3f6a3c25f2b14e09cf7359e00697c4f6260c8cef534afeb77412deb93599995 not found: ID does not exist" Oct 30 18:41:54.484398 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.484360 5070 scope.go:122] "RemoveContainer" containerID="b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661" Oct 30 18:41:54.492110 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.492092 5070 scope.go:122] "RemoveContainer" containerID="c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4" Oct 30 18:41:54.500173 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.500136 5070 scope.go:122] "RemoveContainer" containerID="b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661" Oct 30 18:41:54.500441 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:54.500424 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661\": container with ID starting with b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661 not found: ID does not exist" containerID="b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661" Oct 30 18:41:54.500504 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.500447 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661"} err="failed to get container status \"b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661\": rpc error: code = NotFound desc = could not find container \"b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661\": container with ID starting with b916ac86700880b926f058e05d85855ebeb0e31606293b8cb9df9ff287ce5661 not found: ID does not exist" Oct 30 18:41:54.500504 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.500462 5070 scope.go:122] "RemoveContainer" containerID="c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4" Oct 30 18:41:54.500721 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:54.500703 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4\": container with ID starting with c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4 not found: ID does not exist" containerID="c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4" Oct 30 18:41:54.500772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.500729 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4"} err="failed to get container status \"c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4\": rpc error: code = NotFound desc = could not find container \"c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4\": container with ID starting with c37df4ec09e1938d5b8bf71fbacc7a437635e31741d42ebd427066839d168fd4 not found: ID does not exist" Oct 30 18:41:54.518744 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.518720 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9"] Oct 30 18:41:54.523477 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.523458 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-48afdd9e-openshift-default-6974b885d7-nsql9"] Oct 30 18:41:54.530640 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.530624 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.530697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.530643 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.530697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.530652 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ntmq\" (UniqueName: \"kubernetes.io/projected/bf6f3142-17d0-4d3e-ad3c-13a65477befb-kube-api-access-4ntmq\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.530697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.530664 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.530697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.530672 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf6f3142-17d0-4d3e-ad3c-13a65477befb-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:54.566075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.566052 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59"] Oct 30 18:41:54.572289 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.572271 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-b77c9bba-gw-openshift-default-797f684ffc-j2q59"] Oct 30 18:41:54.618360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.618325 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:54.771512 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.771425 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:54.943312 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:54.943278 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:55.031293 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.031228 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" path="/var/lib/kubelet/pods/62f70766-4490-4d10-a6dd-aaed2ad4cead/volumes" Oct 30 18:41:55.031669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.031654 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" path="/var/lib/kubelet/pods/a7343159-7895-4eb8-8a5b-57d1c28eecef/volumes" Oct 30 18:41:55.032084 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.032071 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" path="/var/lib/kubelet/pods/bf6f3142-17d0-4d3e-ad3c-13a65477befb/volumes" Oct 30 18:41:55.117692 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.117665 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:55.276366 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.276336 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:55.441406 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.441372 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:55.966899 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:55.966864 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:56.060814 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.060791 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:56.132615 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.132586 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:56.151818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.151780 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.151818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.151821 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.151847 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.151883 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.151913 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.151956 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.152019 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.152049 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.152091 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket\") pod \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\" (UID: \"c034bd64-3f9e-4b7a-a2cd-607b853adaae\") " Oct 30 18:41:56.152314 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.152190 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:56.152468 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.152442 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.152810 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.153184 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.153431 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.153450 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.153466 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.153479 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.153562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.153539 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:56.154780 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.154748 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:56.155044 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.154997 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "kube-api-access-9f9t9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:56.155044 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.155025 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:56.155247 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.155102 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo" pod "c034bd64-3f9e-4b7a-a2cd-607b853adaae" (UID: "c034bd64-3f9e-4b7a-a2cd-607b853adaae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:41:56.254353 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.254316 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.254353 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.254349 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f9t9\" (UniqueName: \"kubernetes.io/projected/c034bd64-3f9e-4b7a-a2cd-607b853adaae-kube-api-access-9f9t9\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.254353 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.254359 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.254651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.254368 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.254651 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.254378 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c034bd64-3f9e-4b7a-a2cd-607b853adaae-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:41:56.337556 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.337520 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:56.480233 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.480119 5070 generic.go:356] "Generic (PLEG): container finished" podID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerID="62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b" exitCode=0 Oct 30 18:41:56.480233 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.480182 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" Oct 30 18:41:56.480233 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.480214 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" event={"ID":"c034bd64-3f9e-4b7a-a2cd-607b853adaae","Type":"ContainerDied","Data":"62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b"} Oct 30 18:41:56.480503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.480309 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t" event={"ID":"c034bd64-3f9e-4b7a-a2cd-607b853adaae","Type":"ContainerDied","Data":"3a5168a337c0ca4c28f8414ce840a092487b03cf18aa91bd40ea6825c12cc14c"} Oct 30 18:41:56.480503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.480338 5070 scope.go:122] "RemoveContainer" containerID="62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b" Oct 30 18:41:56.489942 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.489925 5070 scope.go:122] "RemoveContainer" containerID="38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670" Oct 30 18:41:56.490613 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.490590 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:56.498236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.498219 5070 scope.go:122] "RemoveContainer" containerID="62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b" Oct 30 18:41:56.498478 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:56.498457 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b\": container with ID starting with 62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b not found: ID does not exist" containerID="62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b" Oct 30 18:41:56.498526 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.498486 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b"} err="failed to get container status \"62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b\": rpc error: code = NotFound desc = could not find container \"62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b\": container with ID starting with 62aca3f72e0b764ae0595f0dee7f60fdd91c691f8ed8833b0788a42d8419c03b not found: ID does not exist" Oct 30 18:41:56.498526 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.498502 5070 scope.go:122] "RemoveContainer" containerID="38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670" Oct 30 18:41:56.498713 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:41:56.498695 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670\": container with ID starting with 38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670 not found: ID does not exist" containerID="38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670" Oct 30 18:41:56.498745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.498719 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670"} err="failed to get container status \"38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670\": rpc error: code = NotFound desc = could not find container \"38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670\": container with ID starting with 38ae5ac60271c1b759a7cafa40848d8e49e19189ab33c30510331cc28c29e670 not found: ID does not exist" Oct 30 18:41:56.535463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.535441 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t"] Oct 30 18:41:56.547779 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.547755 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ed302a08-openshift-default-545bd55fff-wcb2t"] Oct 30 18:41:56.899692 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:56.899658 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:57.032565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.032533 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" path="/var/lib/kubelet/pods/c034bd64-3f9e-4b7a-a2cd-607b853adaae/volumes" Oct 30 18:41:57.055607 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.055582 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:57.207638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.207562 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:57.354552 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.354518 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:57.480205 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480111 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4"] Oct 30 18:41:57.480590 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480575 5070 cpu_manager.go:414] "containerMap: removing container" podUID="feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" containerName="revoke-keys" Oct 30 18:41:57.480629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480592 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" containerName="revoke-keys" Oct 30 18:41:57.480629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480602 5070 cpu_manager.go:414] "containerMap: removing container" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" Oct 30 18:41:57.480629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480608 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" Oct 30 18:41:57.480629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480620 5070 cpu_manager.go:414] "containerMap: removing container" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" Oct 30 18:41:57.480629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480627 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480636 5070 cpu_manager.go:414] "containerMap: removing container" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480641 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480653 5070 cpu_manager.go:414] "containerMap: removing container" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480658 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480664 5070 cpu_manager.go:414] "containerMap: removing container" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480669 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480677 5070 cpu_manager.go:414] "containerMap: removing container" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480681 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" Oct 30 18:41:57.480785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480781 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="feb56b2f-0472-40c9-ba73-bbc5bbfe13cd" containerName="revoke-keys" Oct 30 18:41:57.481064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480793 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" Oct 30 18:41:57.481064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480801 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" Oct 30 18:41:57.481064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480809 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" Oct 30 18:41:57.481064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480818 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" Oct 30 18:41:57.481064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480825 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="c034bd64-3f9e-4b7a-a2cd-607b853adaae" containerName="istio-proxy" Oct 30 18:41:57.481064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.480836 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" Oct 30 18:41:57.485702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.485680 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.524336 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.524313 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-0ddc26\"" Oct 30 18:41:57.530193 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.530168 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4"] Oct 30 18:41:57.538429 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.538405 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-0ddc26-dockercfg-nggzk\"" Oct 30 18:41:57.566666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.566642 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.566780 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.566711 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drdbp\" (UniqueName: \"kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp\") pod \"payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.628199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.628166 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:57.667709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.667680 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.667874 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.667767 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drdbp\" (UniqueName: \"kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp\") pod \"payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.668493 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.668469 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.703056 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.703026 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drdbp\" (UniqueName: \"kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp\") pod \"payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.710667 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.710640 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc"] Oct 30 18:41:57.711155 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.711128 5070 cpu_manager.go:414] "containerMap: removing container" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" Oct 30 18:41:57.711219 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.711157 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" Oct 30 18:41:57.711219 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.711173 5070 cpu_manager.go:414] "containerMap: removing container" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" Oct 30 18:41:57.711219 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.711180 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62f70766-4490-4d10-a6dd-aaed2ad4cead" containerName="istio-proxy" Oct 30 18:41:57.711352 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.711260 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7343159-7895-4eb8-8a5b-57d1c28eecef" containerName="istio-proxy" Oct 30 18:41:57.711352 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.711268 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf6f3142-17d0-4d3e-ad3c-13a65477befb" containerName="istio-proxy" Oct 30 18:41:57.715607 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.714652 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.730361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.730318 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-a759a1-openshift-default-dockercfg-2pknt\"" Oct 30 18:41:57.741767 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.741745 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc"] Oct 30 18:41:57.768374 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768317 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768555 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768435 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768555 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768496 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768555 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768518 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768685 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768653 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768727 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768703 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whqx9\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768800 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768862 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768847 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.768951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.768937 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.798295 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.798269 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:41:57.809412 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.809392 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:57.877734 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877697 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.877893 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877759 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.877893 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877807 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.877893 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877847 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.877893 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877890 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.878121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877944 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.878121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.877970 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.878121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.878020 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.878121 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.878049 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whqx9\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.879320 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.878964 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.879320 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.879223 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.879320 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.879243 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.880567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.880488 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.881017 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.880976 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.884764 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.884723 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.887561 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.887527 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.911809 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.911776 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.912043 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.912022 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whqx9\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9\") pod \"e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:57.975330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:57.975287 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:58.029784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.029704 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:58.067155 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.067116 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv"] Oct 30 18:41:58.072620 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.072597 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.089082 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.089041 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 18:41:58.089235 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.089106 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-0ddc26-serving-cert\"" Oct 30 18:41:58.089235 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.089193 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bnjrn\"" Oct 30 18:41:58.109307 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:58.109272 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a478c0_037b_4074_a788_b658798a3d8f.slice/crio-37a842910fa4f754e957df2b8e6359b16b4349faf9f6676e1e162d9072e01a2e WatchSource:0}: Error finding container 37a842910fa4f754e957df2b8e6359b16b4349faf9f6676e1e162d9072e01a2e: Status 404 returned error can't find the container with id 37a842910fa4f754e957df2b8e6359b16b4349faf9f6676e1e162d9072e01a2e Oct 30 18:41:58.109719 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.109704 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv"] Oct 30 18:41:58.113558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.113520 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4"] Oct 30 18:41:58.181330 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.181298 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tvf8\" (UniqueName: \"kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.181501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.181416 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.181501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.181448 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.181616 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.181537 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.188069 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.188020 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:58.283188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.283074 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.283188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.283128 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.283695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.283225 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tvf8\" (UniqueName: \"kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.283695 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.283311 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.284042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.284016 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.285715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.285696 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.285804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.285734 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.314666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.314639 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tvf8\" (UniqueName: \"kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8\") pod \"maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.342719 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.340470 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:58.348495 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.348469 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc"] Oct 30 18:41:58.349344 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:58.349319 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1161e03c_ead8_4397_bc7a_57ab809cec75.slice/crio-30a356b02a49d0521c095872475ff39660a7f26ca2166b2724bb0cef992a45bf WatchSource:0}: Error finding container 30a356b02a49d0521c095872475ff39660a7f26ca2166b2724bb0cef992a45bf: Status 404 returned error can't find the container with id 30a356b02a49d0521c095872475ff39660a7f26ca2166b2724bb0cef992a45bf Oct 30 18:41:58.351776 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.351745 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:58.351850 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.351807 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:58.351850 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.351836 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:41:58.390963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.390941 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:41:58.494501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.494451 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" event={"ID":"1161e03c-ead8-4397-bc7a-57ab809cec75","Type":"ContainerStarted","Data":"1eb6efadf631c4dc77bfa425aa256ecb3b76b54ad58e61e49cd107a3d6fb28ab"} Oct 30 18:41:58.494773 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.494735 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" event={"ID":"1161e03c-ead8-4397-bc7a-57ab809cec75","Type":"ContainerStarted","Data":"30a356b02a49d0521c095872475ff39660a7f26ca2166b2724bb0cef992a45bf"} Oct 30 18:41:58.496215 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.496171 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" event={"ID":"e4a478c0-037b-4074-a788-b658798a3d8f","Type":"ContainerStarted","Data":"37a842910fa4f754e957df2b8e6359b16b4349faf9f6676e1e162d9072e01a2e"} Oct 30 18:41:58.540093 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.539992 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podStartSLOduration=1.539977106 podStartE2EDuration="1.539977106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:41:58.539667774 +0000 UTC m=+635.654801918" watchObservedRunningTime="2026-10-30 18:41:58.539977106 +0000 UTC m=+635.655111022" Oct 30 18:41:58.706943 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:58.706519 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv"] Oct 30 18:41:58.723111 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:41:58.723077 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b89d29_0142_4973_a8bd_a39808ca2ec1.slice/crio-7e2687dfb93e5d70fff520623f83f95d7aa6bc76362591b9bf2fee9d1f0fb1cd WatchSource:0}: Error finding container 7e2687dfb93e5d70fff520623f83f95d7aa6bc76362591b9bf2fee9d1f0fb1cd: Status 404 returned error can't find the container with id 7e2687dfb93e5d70fff520623f83f95d7aa6bc76362591b9bf2fee9d1f0fb1cd Oct 30 18:41:59.033734 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:59.033708 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:59.036521 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:59.036475 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:59.516462 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:59.516423 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" event={"ID":"f5b89d29-0142-4973-a8bd-a39808ca2ec1","Type":"ContainerStarted","Data":"7e2687dfb93e5d70fff520623f83f95d7aa6bc76362591b9bf2fee9d1f0fb1cd"} Oct 30 18:41:59.516897 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:59.516488 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:59.517466 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:59.517383 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:41:59.867863 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:41:59.867823 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:00.036567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:00.036529 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:00.782377 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:00.782346 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:00.951123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:00.951084 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:01.385608 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.385576 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:01.529553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.529512 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" event={"ID":"f5b89d29-0142-4973-a8bd-a39808ca2ec1","Type":"ContainerStarted","Data":"3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923"} Oct 30 18:42:01.529733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.529583 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:42:01.531118 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.531093 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" event={"ID":"e4a478c0-037b-4074-a788-b658798a3d8f","Type":"ContainerStarted","Data":"61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d"} Oct 30 18:42:01.531296 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.531281 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:42:01.582626 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.582532 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:01.585862 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.585807 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" podStartSLOduration=1.007162256 podStartE2EDuration="3.585792075s" totalImagesPullingTime="2.578629819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:01.58409111 +0000 UTC m=+638.699225041" watchObservedRunningTime="2026-10-30 18:42:01.585792075 +0000 UTC m=+638.700925991" Oct 30 18:42:01.632551 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:01.632491 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" podStartSLOduration=2.156259882 podStartE2EDuration="4.632478701s" totalImagesPullingTime="2.476218819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:01.630075951 +0000 UTC m=+638.745209878" watchObservedRunningTime="2026-10-30 18:42:01.632478701 +0000 UTC m=+638.747612649" Oct 30 18:42:02.108061 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:02.108022 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:02.323384 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:02.323349 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:03.833096 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:42:03.833060 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:42:04.250731 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.250689 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd"] Oct 30 18:42:04.254699 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.254677 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.283927 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.283892 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd"] Oct 30 18:42:04.336373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.336341 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-f6f7d5-serving-cert\"" Oct 30 18:42:04.365435 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.365398 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.365593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.365451 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.365593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.365559 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.365926 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.365626 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtc4\" (UniqueName: \"kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.404524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.404483 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:04.467311 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.467274 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtc4\" (UniqueName: \"kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.467489 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.467386 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.467489 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.467424 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.467489 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.467445 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.468233 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.468214 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.469921 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.469900 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.469987 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.469939 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.492159 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.492123 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtc4\" (UniqueName: \"kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4\") pod \"maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.567831 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.567758 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:04.569331 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.569308 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:04.891779 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:04.891748 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd"] Oct 30 18:42:04.892127 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:42:04.892089 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702ca51e_a16f_441b_a169_b29fb5789b48.slice/crio-8eed99b84285810d530de5ce8f54e775103abb279b18eeff4ad80275a71e6188 WatchSource:0}: Error finding container 8eed99b84285810d530de5ce8f54e775103abb279b18eeff4ad80275a71e6188: Status 404 returned error can't find the container with id 8eed99b84285810d530de5ce8f54e775103abb279b18eeff4ad80275a71e6188 Oct 30 18:42:05.556932 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:05.556849 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" event={"ID":"702ca51e-a16f-441b-a169-b29fb5789b48","Type":"ContainerStarted","Data":"a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc"} Oct 30 18:42:05.556932 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:05.556903 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" event={"ID":"702ca51e-a16f-441b-a169-b29fb5789b48","Type":"ContainerStarted","Data":"8eed99b84285810d530de5ce8f54e775103abb279b18eeff4ad80275a71e6188"} Oct 30 18:42:05.557136 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:05.557066 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:05.602402 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:05.602351 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" podStartSLOduration=1.21261354 podStartE2EDuration="1.60233556s" totalImagesPullingTime="389.72202ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:05.601183292 +0000 UTC m=+642.716317207" watchObservedRunningTime="2026-10-30 18:42:05.60233556 +0000 UTC m=+642.717469477" Oct 30 18:42:06.212947 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:06.212910 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:06.389864 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:06.389832 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:07.544089 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:07.544061 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:42:07.994382 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:07.994344 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:08.224352 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:08.224318 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:08.652406 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:08.652369 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:08.826636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:08.826599 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:10.123320 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:10.123289 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:10.325779 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:10.325711 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:10.595243 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:10.595141 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:11.567759 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:11.567729 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:42:12.542642 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:12.542607 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:42:12.669342 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:12.669312 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:13.046420 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:13.046384 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:13.702874 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:13.702842 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:13.855938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:13.855902 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:14.000906 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:14.000823 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:14.205832 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:14.205793 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:14.664846 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:14.664814 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:14.866347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:14.866295 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:15.035764 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.035672 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:15.208860 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.208818 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r"] Oct 30 18:42:15.215461 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.215435 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.233821 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.233775 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:15.245032 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.245009 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"openshift-service-ca.crt\"" Oct 30 18:42:15.245206 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.245009 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"kube-root-ca.crt\"" Oct 30 18:42:15.245284 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.245233 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"simulator-w4-a47385-kserve-self-signed-certs\"" Oct 30 18:42:15.245702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.245679 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-a47385\"/\"default-dockercfg-658qm\"" Oct 30 18:42:15.247425 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.247402 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r"] Oct 30 18:42:15.387304 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.387269 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.387483 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.387412 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.387483 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.387440 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.387483 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.387461 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd7xv\" (UniqueName: \"kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.387601 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.387484 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.387601 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.387548 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.488505 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.488473 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.488653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.488561 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.488653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.488579 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.488653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.488595 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd7xv\" (UniqueName: \"kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.488653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.488614 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.488653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.488635 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.489029 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.489007 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.489090 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.489038 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.489090 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.489068 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.490715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.490694 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.490994 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.490976 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.501222 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.501195 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:15.513247 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.513226 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd7xv\" (UniqueName: \"kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv\") pod \"simulator-w4-a47385-kserve-56867dbd67-8ms9r\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.529768 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.529743 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:15.843123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:15.843091 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r"] Oct 30 18:42:15.843781 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:42:15.843745 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5cfe666_a504_4b72_8fa2_ea8403695758.slice/crio-7f395ebecf3fed5b72f0f99d9cd1707a6ff48f674957182367217e0b5cf361db WatchSource:0}: Error finding container 7f395ebecf3fed5b72f0f99d9cd1707a6ff48f674957182367217e0b5cf361db: Status 404 returned error can't find the container with id 7f395ebecf3fed5b72f0f99d9cd1707a6ff48f674957182367217e0b5cf361db Oct 30 18:42:16.346228 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:16.346189 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:16.646964 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:16.646928 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" event={"ID":"c5cfe666-a504-4b72-8fa2-ea8403695758","Type":"ContainerStarted","Data":"b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93"} Oct 30 18:42:16.647190 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:16.646970 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" event={"ID":"c5cfe666-a504-4b72-8fa2-ea8403695758","Type":"ContainerStarted","Data":"7f395ebecf3fed5b72f0f99d9cd1707a6ff48f674957182367217e0b5cf361db"} Oct 30 18:42:16.691854 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:16.691806 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" podStartSLOduration=1.691793005 podStartE2EDuration="1.691793005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:16.691663547 +0000 UTC m=+653.806797461" watchObservedRunningTime="2026-10-30 18:42:16.691793005 +0000 UTC m=+653.806926922" Oct 30 18:42:16.888216 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:16.888183 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:17.535320 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:17.535247 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:18.966240 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:18.966199 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:20.209029 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:20.208996 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:20.564967 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:20.564888 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:20.873345 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:20.873311 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:21.044365 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:21.044332 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:21.249505 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:21.249430 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:21.438567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:21.438534 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:22.019728 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:22.019690 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:22.210123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:22.210089 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:23.075778 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:23.075721 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:24.942872 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:24.942810 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:25.529987 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.529951 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:25.542657 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.542630 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:25.699267 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.699232 5070 generic.go:356] "Generic (PLEG): container finished" podID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerID="1eb6efadf631c4dc77bfa425aa256ecb3b76b54ad58e61e49cd107a3d6fb28ab" exitCode=0 Oct 30 18:42:25.699463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.699310 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" event={"ID":"1161e03c-ead8-4397-bc7a-57ab809cec75","Type":"ContainerDied","Data":"1eb6efadf631c4dc77bfa425aa256ecb3b76b54ad58e61e49cd107a3d6fb28ab"} Oct 30 18:42:25.699820 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.699801 5070 scope.go:122] "RemoveContainer" containerID="1eb6efadf631c4dc77bfa425aa256ecb3b76b54ad58e61e49cd107a3d6fb28ab" Oct 30 18:42:25.701011 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.700980 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:42:25.701109 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.701063 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:42:25.701201 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.701115 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:42:25.711042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:25.711023 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:42:26.029002 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.028951 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2"] Oct 30 18:42:26.032282 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.032258 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.050523 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.050481 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"default-dockercfg-vnsmm\"" Oct 30 18:42:26.050650 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.050625 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"openshift-service-ca.crt\"" Oct 30 18:42:26.050741 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.050724 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"simulator-w3-0ddc26-kserve-self-signed-certs\"" Oct 30 18:42:26.050796 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.050765 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"kube-root-ca.crt\"" Oct 30 18:42:26.053725 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.053705 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2"] Oct 30 18:42:26.124704 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.124673 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.124862 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.124729 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.124862 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.124839 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.124938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.124887 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.124971 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.124932 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cgcb\" (UniqueName: \"kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.124971 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.124963 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.140976 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.140922 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:26.225986 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.225948 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226012 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226057 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226076 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cgcb\" (UniqueName: \"kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226097 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226179 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226537 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226458 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226537 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226520 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.226654 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.226634 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.228431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.228408 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.228685 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.228661 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.250185 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.250141 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cgcb\" (UniqueName: \"kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb\") pod \"simulator-w3-0ddc26-kserve-6978bb9469-wh9h2\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.334939 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.322977 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:26.352735 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.352411 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:26.661493 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.661468 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2"] Oct 30 18:42:26.662593 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:42:26.662564 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70a677ad_6ac6_41a2_bb65_923f670c7048.slice/crio-2fa3578334b1cdeaa3fe4b337acb57de75083dbf7da28f684f7353583a0e8b76 WatchSource:0}: Error finding container 2fa3578334b1cdeaa3fe4b337acb57de75083dbf7da28f684f7353583a0e8b76: Status 404 returned error can't find the container with id 2fa3578334b1cdeaa3fe4b337acb57de75083dbf7da28f684f7353583a0e8b76 Oct 30 18:42:26.707098 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.707067 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" event={"ID":"1161e03c-ead8-4397-bc7a-57ab809cec75","Type":"ContainerStarted","Data":"cd5dff9d6d3a5492946d44d308b155b77960e4d8d4ffd13bc70550057a991288"} Oct 30 18:42:26.708908 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.708882 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" event={"ID":"70a677ad-6ac6-41a2-bb65-923f670c7048","Type":"ContainerStarted","Data":"2fa3578334b1cdeaa3fe4b337acb57de75083dbf7da28f684f7353583a0e8b76"} Oct 30 18:42:26.879954 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:26.879921 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:27.039774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.039681 5070 patch_prober.go:29] interesting pod/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 30 18:42:27.039774 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.039737 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 30 18:42:27.042033 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.042013 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:42:27.060434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.060409 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:27.238791 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.238752 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:27.739907 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.739855 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" event={"ID":"70a677ad-6ac6-41a2-bb65-923f670c7048","Type":"ContainerStarted","Data":"d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039"} Oct 30 18:42:27.795066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:27.795017 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" podStartSLOduration=2.795002737 podStartE2EDuration="2.795002737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:27.792502592 +0000 UTC m=+664.907636528" watchObservedRunningTime="2026-10-30 18:42:27.795002737 +0000 UTC m=+664.910136653" Oct 30 18:42:28.036226 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:28.035601 5070 patch_prober.go:29] interesting pod/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 30 18:42:28.036226 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:28.035666 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 30 18:42:28.353805 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:28.353715 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:28.691310 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:28.691270 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:28.845112 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:28.845001 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:29.035221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:29.034511 5070 patch_prober.go:29] interesting pod/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 30 18:42:29.035221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:29.034572 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 30 18:42:29.538170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:29.524231 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:42:30.038310 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:30.038266 5070 patch_prober.go:29] interesting pod/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 30 18:42:30.038499 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:30.038333 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 30 18:42:30.436183 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:30.432705 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:31.033289 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:31.033249 5070 patch_prober.go:29] interesting pod/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 30 18:42:31.033882 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:31.033851 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 30 18:42:32.034321 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:32.034290 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:42:32.035295 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:32.035273 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:42:32.044732 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:32.044529 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:33.733047 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:33.733010 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:33.832889 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:42:33.832852 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:42:34.152163 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:34.152118 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:35.337824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:35.337748 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:36.353582 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:36.353551 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:36.366649 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:36.366616 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:36.423496 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:36.423461 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:36.573098 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:36.573063 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:36.793536 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:36.793502 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:42:36.980042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:36.979997 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:38.729822 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.729790 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:38.827588 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.827559 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq"] Oct 30 18:42:38.831412 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.831388 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:38.846883 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.846849 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-a759a1\"/\"openshift-service-ca.crt\"" Oct 30 18:42:38.859586 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.859559 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a759a1\"/\"simulator-w1-a759a1-kserve-self-signed-certs\"" Oct 30 18:42:38.859721 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.859591 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-a759a1\"/\"kube-root-ca.crt\"" Oct 30 18:42:38.861884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.861860 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq"] Oct 30 18:42:38.938666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.938613 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:38.947216 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.947196 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a759a1\"/\"default-dockercfg-mm2jl\"" Oct 30 18:42:38.969004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.968979 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:38.969095 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.969010 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:38.969095 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.969050 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:38.969235 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.969215 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:38.969286 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.969260 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:38.969322 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:38.969312 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g76zp\" (UniqueName: \"kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.070622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070538 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.070622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070592 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.070841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070631 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76zp\" (UniqueName: \"kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.070841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070685 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.070841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070711 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.070841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070762 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.071038 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070980 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.071038 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.070996 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.071176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.071055 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.072908 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.072887 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.073196 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.073179 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.098487 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.098452 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76zp\" (UniqueName: \"kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp\") pod \"simulator-w1-a759a1-kserve-665b7bc764-l85lq\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.147937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.147905 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:39.446624 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.446598 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq"] Oct 30 18:42:39.447646 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:42:39.447620 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f01e1bb_39a4_4233_a132_6f5f2c024dac.slice/crio-b9ee8cc5ee2aab26aa3baa79f10988089b384b0f0a91ac0a462d1e8fdecf373c WatchSource:0}: Error finding container b9ee8cc5ee2aab26aa3baa79f10988089b384b0f0a91ac0a462d1e8fdecf373c: Status 404 returned error can't find the container with id b9ee8cc5ee2aab26aa3baa79f10988089b384b0f0a91ac0a462d1e8fdecf373c Oct 30 18:42:39.801697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.801615 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" event={"ID":"0f01e1bb-39a4-4233-a132-6f5f2c024dac","Type":"ContainerStarted","Data":"6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf"} Oct 30 18:42:39.801697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.801667 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" event={"ID":"0f01e1bb-39a4-4233-a132-6f5f2c024dac","Type":"ContainerStarted","Data":"b9ee8cc5ee2aab26aa3baa79f10988089b384b0f0a91ac0a462d1e8fdecf373c"} Oct 30 18:42:39.844928 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:39.844877 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" podStartSLOduration=1.844864217 podStartE2EDuration="1.844864217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:39.843530772 +0000 UTC m=+676.958664670" watchObservedRunningTime="2026-10-30 18:42:39.844864217 +0000 UTC m=+676.959998130" Oct 30 18:42:40.148394 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:40.148362 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:40.316824 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:40.316785 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:41.835449 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:41.835410 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:43.452284 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:43.452245 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:45.050238 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:45.050203 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:45.224325 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:45.224293 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:46.858844 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:46.858814 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:48.417761 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:48.417723 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:49.149032 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:49.148996 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:49.161553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:49.161532 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:49.861656 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:49.861627 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:42:50.037120 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:50.037090 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:51.678324 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:51.678294 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:53.329114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:53.329079 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:54.661924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:54.661874 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:55.317029 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:55.316996 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:56.977929 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:56.977893 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:58.322194 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:58.322154 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:58.483464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:58.483430 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:58.648479 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:42:58.648445 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:00.299961 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:00.299926 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:02.043060 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:02.043028 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:03.742181 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:03.742128 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:03.832352 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:43:03.832317 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:43:05.352715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:05.352669 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:07.054196 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:07.054159 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:07.434300 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:07.434267 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:08.741841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:08.741809 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:10.438016 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.437950 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:10.691731 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.691648 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7"] Oct 30 18:43:10.695339 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.695315 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.725872 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.725840 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a759a1\"/\"premium-w1-a759a1-kserve-self-signed-certs\"" Oct 30 18:43:10.728010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.727989 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7"] Oct 30 18:43:10.789763 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.789731 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:10.796835 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.796811 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.796923 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.796843 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmtbf\" (UniqueName: \"kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.796967 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.796923 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.797007 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.796970 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.797007 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.796993 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.797084 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.797055 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.897752 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.897705 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.897963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.897798 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.897963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.897831 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.897963 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.897854 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.898124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.897967 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.898124 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.898000 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmtbf\" (UniqueName: \"kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.898293 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.898197 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.898644 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.898615 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.898772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.898659 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.900694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.900665 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.901374 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.901351 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:10.922135 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:10.922106 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmtbf\" (UniqueName: \"kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf\") pod \"premium-w1-a759a1-kserve-785d47b9d5-j9dp7\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:11.013306 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:11.013229 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:11.313948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:11.313920 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7"] Oct 30 18:43:11.314393 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:43:11.314371 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f60b6bc_6a69_47e6_bb5b_545aec37fd45.slice/crio-99c543f5988ac75f9dc9ae5bd96f29713ac4b9520fa21b7f0395db33e199d626 WatchSource:0}: Error finding container 99c543f5988ac75f9dc9ae5bd96f29713ac4b9520fa21b7f0395db33e199d626: Status 404 returned error can't find the container with id 99c543f5988ac75f9dc9ae5bd96f29713ac4b9520fa21b7f0395db33e199d626 Oct 30 18:43:11.967924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:11.967886 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" event={"ID":"7f60b6bc-6a69-47e6-bb5b-545aec37fd45","Type":"ContainerStarted","Data":"742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868"} Oct 30 18:43:11.968338 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:11.967933 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" event={"ID":"7f60b6bc-6a69-47e6-bb5b-545aec37fd45","Type":"ContainerStarted","Data":"99c543f5988ac75f9dc9ae5bd96f29713ac4b9520fa21b7f0395db33e199d626"} Oct 30 18:43:12.013340 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:12.013310 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:12.017344 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:12.017306 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" podStartSLOduration=2.017294908 podStartE2EDuration="2.017294908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:12.015244959 +0000 UTC m=+709.130378875" watchObservedRunningTime="2026-10-30 18:43:12.017294908 +0000 UTC m=+709.132428824" Oct 30 18:43:12.057419 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:12.057391 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:13.291049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:13.291013 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:13.835076 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:13.835039 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:15.444844 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:15.444808 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:16.475397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:16.475329 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:17.394263 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:17.394225 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:18.978263 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:18.978233 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:20.619434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:20.619398 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:20.777605 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:20.777577 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:21.013518 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:21.013439 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:21.037767 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:21.034775 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:22.025230 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:22.025205 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:43:22.416277 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:22.416239 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:24.037547 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:24.037514 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:25.763228 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:25.763186 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:27.447160 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:27.447100 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:29.068310 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:29.068279 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:30.757253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:30.757207 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:32.353979 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:32.353945 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:33.832182 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:43:33.832132 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:43:34.016877 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:34.016831 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:34.501878 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:34.501848 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:34.814220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:34.814110 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:35.624705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:35.624665 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:37.334718 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:37.334646 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:39.012687 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:39.012657 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:40.891504 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:40.891472 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:42.677366 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:42.677335 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:44.559782 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:44.559749 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:46.138101 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:46.138046 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:47.733347 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:47.733314 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:49.345130 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:49.345100 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:50.934171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:50.934125 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:52.566458 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.566418 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph"] Oct 30 18:43:52.571363 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.571341 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.591232 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.591198 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph"] Oct 30 18:43:52.600523 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.600499 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f6f7d5-openshift-default-dockercfg-9sw8n\"" Oct 30 18:43:52.675050 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.675016 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:52.720742 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720707 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.720911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720751 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.720911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720831 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.720911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720854 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.720911 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720891 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.721056 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720938 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.721056 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.720996 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbhsg\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.721056 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.721049 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.721183 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.721117 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822482 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822390 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822482 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822458 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822706 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822511 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822706 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822542 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822706 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822570 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822706 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822598 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822902 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822751 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822902 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822812 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.822902 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822848 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbhsg\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.823061 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.822963 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.823120 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.823054 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.823226 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.823208 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.823278 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.823250 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.823342 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.823326 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.825346 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.825322 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.825468 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.825451 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.847038 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.847010 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.849970 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.849950 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbhsg\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg\") pod \"e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:52.885049 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:52.885030 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:53.205300 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:53.205271 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph"] Oct 30 18:43:53.205577 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:43:53.205548 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c98abf2_a117_444a_b125_324f6c83bad7.slice/crio-f1c13e51f31a7b3ff99739028d87ab53100a44870d08e5c13442af0aa414ae4c WatchSource:0}: Error finding container f1c13e51f31a7b3ff99739028d87ab53100a44870d08e5c13442af0aa414ae4c: Status 404 returned error can't find the container with id f1c13e51f31a7b3ff99739028d87ab53100a44870d08e5c13442af0aa414ae4c Oct 30 18:43:53.207741 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:53.207708 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:43:53.207833 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:53.207768 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:43:53.207833 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:53.207796 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:43:54.188766 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:54.188726 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" event={"ID":"5c98abf2-a117-444a-b125-324f6c83bad7","Type":"ContainerStarted","Data":"07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3"} Oct 30 18:43:54.189186 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:54.188825 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" event={"ID":"5c98abf2-a117-444a-b125-324f6c83bad7","Type":"ContainerStarted","Data":"f1c13e51f31a7b3ff99739028d87ab53100a44870d08e5c13442af0aa414ae4c"} Oct 30 18:43:54.344804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:54.337586 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:54.885736 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:54.885699 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:54.887261 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:54.887238 5070 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 18:43:54.887373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:54.887291 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 18:43:55.885414 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:55.885377 5070 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 18:43:55.885838 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:55.885450 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 18:43:55.982522 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:55.982481 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:56.886050 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:56.886019 5070 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 18:43:56.886448 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:56.886079 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 18:43:57.864020 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:57.863985 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:57.885684 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:57.885660 5070 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 18:43:57.885787 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:57.885713 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 18:43:58.886444 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:58.886408 5070 patch_prober.go:29] interesting pod/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 18:43:58.886937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:58.886458 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 18:43:59.579481 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:59.579448 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:59.889902 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:59.889874 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:59.890372 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:59.890350 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:59.891075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:59.891055 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:43:59.951377 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:43:59.951332 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podStartSLOduration=7.951319637 podStartE2EDuration="7.951319637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:54.264502898 +0000 UTC m=+751.379636814" watchObservedRunningTime="2026-10-30 18:43:59.951319637 +0000 UTC m=+757.066453553" Oct 30 18:44:01.173419 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.173380 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:01.218301 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.218273 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9"] Oct 30 18:44:01.222275 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.222251 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.239187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.239160 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:44:01.239187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.239187 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:44:01.253969 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.253943 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9"] Oct 30 18:44:01.323426 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.323393 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.323426 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.323426 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs7pj\" (UniqueName: \"kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.323661 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.323619 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.424648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.424556 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.424648 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.424618 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs7pj\" (UniqueName: \"kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.424854 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.424704 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.425335 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.425312 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.426987 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.426959 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.449584 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.449558 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs7pj\" (UniqueName: \"kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj\") pod \"maas-api-revoke-sub-5022a55d-lbfp9\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.537091 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.537058 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:01.843210 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:01.839513 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9"] Oct 30 18:44:01.843210 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:44:01.840816 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6409e0_4b88_4ebc_8ef2_22dad5bfc027.slice/crio-6f6ad937da0f5f50cec571ba71a78ab06c1137d0503cf6c76e731af2b3b3962d WatchSource:0}: Error finding container 6f6ad937da0f5f50cec571ba71a78ab06c1137d0503cf6c76e731af2b3b3962d: Status 404 returned error can't find the container with id 6f6ad937da0f5f50cec571ba71a78ab06c1137d0503cf6c76e731af2b3b3962d Oct 30 18:44:02.241638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:02.241597 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" event={"ID":"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027","Type":"ContainerStarted","Data":"eca20a47a4ae42b800ed05b9ebd99036d891deb45ee3bfbf3115f8fa4858d0c8"} Oct 30 18:44:02.242086 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:02.241675 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" event={"ID":"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027","Type":"ContainerStarted","Data":"6f6ad937da0f5f50cec571ba71a78ab06c1137d0503cf6c76e731af2b3b3962d"} Oct 30 18:44:02.285449 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:02.285393 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" podStartSLOduration=1.285380403 podStartE2EDuration="1.285380403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:02.283202871 +0000 UTC m=+759.398336788" watchObservedRunningTime="2026-10-30 18:44:02.285380403 +0000 UTC m=+759.400514319" Oct 30 18:44:02.816008 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:02.815978 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:03.249593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:03.249555 5070 generic.go:356] "Generic (PLEG): container finished" podID="ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" containerID="eca20a47a4ae42b800ed05b9ebd99036d891deb45ee3bfbf3115f8fa4858d0c8" exitCode=0 Oct 30 18:44:03.249977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:03.249631 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" event={"ID":"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027","Type":"ContainerDied","Data":"eca20a47a4ae42b800ed05b9ebd99036d891deb45ee3bfbf3115f8fa4858d0c8"} Oct 30 18:44:03.832867 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:44:03.832833 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:44:04.347088 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.347038 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:04.402496 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.402470 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:04.461102 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.461073 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca\") pod \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " Oct 30 18:44:04.461307 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.461158 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj\" (UniqueName: \"kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj\") pod \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " Oct 30 18:44:04.461376 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.461322 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca\") pod \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\" (UID: \"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027\") " Oct 30 18:44:04.461458 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.461434 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca" pod "ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" (UID: "ff6409e0-4b88-4ebc-8ef2-22dad5bfc027"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:44:04.461696 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.461676 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-openshift-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.463363 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.463335 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca" pod "ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" (UID: "ff6409e0-4b88-4ebc-8ef2-22dad5bfc027"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:44:04.463462 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.463404 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj" pod "ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" (UID: "ff6409e0-4b88-4ebc-8ef2-22dad5bfc027"). InnerVolumeSpecName "kube-api-access-rs7pj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:44:04.562685 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.562611 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-maas-api-xks-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:44:04.562685 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:04.562639 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs7pj\" (UniqueName: \"kubernetes.io/projected/ff6409e0-4b88-4ebc-8ef2-22dad5bfc027-kube-api-access-rs7pj\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:44:05.260971 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.260940 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" Oct 30 18:44:05.261140 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.260962 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5022a55d-lbfp9" event={"ID":"ff6409e0-4b88-4ebc-8ef2-22dad5bfc027","Type":"ContainerDied","Data":"6f6ad937da0f5f50cec571ba71a78ab06c1137d0503cf6c76e731af2b3b3962d"} Oct 30 18:44:05.261140 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.261016 5070 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f6ad937da0f5f50cec571ba71a78ab06c1137d0503cf6c76e731af2b3b3962d" Oct 30 18:44:05.578533 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.578498 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv"] Oct 30 18:44:05.579805 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.579771 5070 cpu_manager.go:414] "containerMap: removing container" podUID="ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" containerName="revoke-keys" Oct 30 18:44:05.579805 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.579801 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" containerName="revoke-keys" Oct 30 18:44:05.579969 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.579955 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff6409e0-4b88-4ebc-8ef2-22dad5bfc027" containerName="revoke-keys" Oct 30 18:44:05.583712 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.583692 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.613599 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.613572 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-0ddc26-openshift-default-dockercfg-sc7gm\"" Oct 30 18:44:05.618010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.617983 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv"] Oct 30 18:44:05.674744 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.674716 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45wgl\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.674895 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.674757 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.674895 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.674785 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.674984 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.674906 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.674984 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.674956 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.675077 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.675059 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.675126 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.675110 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.675191 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.675175 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.675244 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.675228 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.775785 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775758 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.775938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775801 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.775938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775833 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45wgl\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.775938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775858 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.775938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775878 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.775938 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775920 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.775956 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776048 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776213 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776090 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776395 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776358 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776393 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776523 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776467 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776734 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776692 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.776850 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.776827 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.778312 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.778286 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.778401 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.778385 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.800036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.800014 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.801586 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.801568 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45wgl\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl\") pod \"e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:05.896577 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:05.896551 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:06.213551 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:06.213525 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv"] Oct 30 18:44:06.214240 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:44:06.214213 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622b1121_a027_4243_be20_75eb8da3137b.slice/crio-a58e24dfbf528a85eaba8bd25c1aae4685c696e69e6a4181eac7f85bb06d2e1e WatchSource:0}: Error finding container a58e24dfbf528a85eaba8bd25c1aae4685c696e69e6a4181eac7f85bb06d2e1e: Status 404 returned error can't find the container with id a58e24dfbf528a85eaba8bd25c1aae4685c696e69e6a4181eac7f85bb06d2e1e Oct 30 18:44:06.216510 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:06.216475 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:06.216607 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:06.216551 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:06.216607 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:06.216594 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:06.267388 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:06.267359 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" event={"ID":"622b1121-a027-4243-be20-75eb8da3137b","Type":"ContainerStarted","Data":"a58e24dfbf528a85eaba8bd25c1aae4685c696e69e6a4181eac7f85bb06d2e1e"} Oct 30 18:44:06.292383 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:06.292345 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:07.274377 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:07.274338 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" event={"ID":"622b1121-a027-4243-be20-75eb8da3137b","Type":"ContainerStarted","Data":"7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5"} Oct 30 18:44:07.348937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:07.348880 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podStartSLOduration=2.348865855 podStartE2EDuration="2.348865855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:07.33459227 +0000 UTC m=+764.449726186" watchObservedRunningTime="2026-10-30 18:44:07.348865855 +0000 UTC m=+764.463999853" Oct 30 18:44:07.897021 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:07.896988 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:07.898657 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:07.898632 5070 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 18:44:07.898772 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:07.898678 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 18:44:07.964235 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:07.964187 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:08.896887 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:08.896853 5070 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 18:44:08.897303 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:08.896903 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 18:44:09.551435 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:09.550631 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:09.897209 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:09.897181 5070 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 18:44:09.897590 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:09.897228 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 18:44:10.897477 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:10.897441 5070 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 18:44:10.897863 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:10.897495 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 18:44:11.236917 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:11.236823 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:11.897209 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:11.897174 5070 patch_prober.go:29] interesting pod/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 18:44:11.897375 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:11.897233 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 18:44:12.900901 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:12.900871 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:12.901421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:12.901236 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:12.901975 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:12.901956 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:44:12.944617 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:12.944574 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:14.648960 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:14.648925 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:15.429308 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:15.429276 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:15.579195 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:15.579140 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:16.266220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:16.266182 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:16.605327 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:16.605247 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:16.812804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:16.812759 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:17.970360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:17.970327 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:19.660091 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:19.660063 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:21.358472 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:21.358438 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:21.758497 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:21.758462 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:23.067723 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:23.067688 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:24.747792 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:24.747751 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:26.692231 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:26.692197 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:28.445211 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:28.445178 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:30.156688 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:30.156648 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:31.833603 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:31.833571 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:33.439259 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:33.439222 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:33.832468 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:44:33.832389 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:44:35.048318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:35.048283 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:36.764597 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:36.764561 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:38.331674 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.331638 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:38.808482 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.808447 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7"] Oct 30 18:44:38.812316 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.812294 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.857716 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.857678 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5"] Oct 30 18:44:38.861841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.861815 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7"] Oct 30 18:44:38.861967 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.861935 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.891257 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.891210 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.891538 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.891511 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.891870 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.891840 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwg7s\" (UniqueName: \"kubernetes.io/projected/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-kube-api-access-nwg7s\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.903465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.891905 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.903465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.891936 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.903465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.891981 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.903465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.892024 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.903465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.892054 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.903465 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.892078 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.915306 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.915279 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5"] Oct 30 18:44:38.993069 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993034 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwg7s\" (UniqueName: \"kubernetes.io/projected/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-kube-api-access-nwg7s\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993255 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993086 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993255 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993112 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993255 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993167 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993255 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993208 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993255 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993234 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993257 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993295 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993346 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993383 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m47dm\" (UniqueName: \"kubernetes.io/projected/7e87a8d3-748e-4674-9da3-e54a3ce24b87-kube-api-access-m47dm\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993425 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993550 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993574 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993609 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.993643 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993639 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.994053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993672 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:38.994053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993721 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.994053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993793 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.994053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993882 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.994053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.993891 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.994419 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.994333 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.994577 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.994559 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.994688 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.994660 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.996431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.996401 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:38.996991 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:38.996970 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:39.037599 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.037577 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwg7s\" (UniqueName: \"kubernetes.io/projected/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-kube-api-access-nwg7s\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:39.039256 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.039232 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d625d9d-56f7-4149-b9e2-5a8b5f114aaf-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-tzpc7\" (UID: \"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:39.094808 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.094749 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.094808 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.094798 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.094940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.094818 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m47dm\" (UniqueName: \"kubernetes.io/projected/7e87a8d3-748e-4674-9da3-e54a3ce24b87-kube-api-access-m47dm\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095265 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095235 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095274 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095304 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095371 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095330 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095541 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095391 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095541 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095440 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095676 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095584 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095744 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095695 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095806 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095747 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.095967 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095946 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.096012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.095951 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.097344 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.097323 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.097421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.097344 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.118335 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.118316 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e87a8d3-748e-4674-9da3-e54a3ce24b87-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.118674 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.118655 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m47dm\" (UniqueName: \"kubernetes.io/projected/7e87a8d3-748e-4674-9da3-e54a3ce24b87-kube-api-access-m47dm\") pod \"maas-default-gateway-openshift-default-6dcf757899-cvhp5\" (UID: \"7e87a8d3-748e-4674-9da3-e54a3ce24b87\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.130714 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.130699 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:39.174797 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.174771 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:39.493659 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:44:39.493630 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d625d9d_56f7_4149_b9e2_5a8b5f114aaf.slice/crio-1def82acab7e1584988e17318ceab69019c59a7c7c5a6b0eff2b8aeaeedcf3de WatchSource:0}: Error finding container 1def82acab7e1584988e17318ceab69019c59a7c7c5a6b0eff2b8aeaeedcf3de: Status 404 returned error can't find the container with id 1def82acab7e1584988e17318ceab69019c59a7c7c5a6b0eff2b8aeaeedcf3de Oct 30 18:44:39.493996 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.493674 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7"] Oct 30 18:44:39.495940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.495903 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.496036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.495967 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.496036 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.495995 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.500002 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.499976 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5"] Oct 30 18:44:39.506173 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:44:39.506133 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e87a8d3_748e_4674_9da3_e54a3ce24b87.slice/crio-6cc5b0e96b699b02b2f925e776a11d9d23bde8ac140f5999cace3bd36d8f35e9 WatchSource:0}: Error finding container 6cc5b0e96b699b02b2f925e776a11d9d23bde8ac140f5999cace3bd36d8f35e9: Status 404 returned error can't find the container with id 6cc5b0e96b699b02b2f925e776a11d9d23bde8ac140f5999cace3bd36d8f35e9 Oct 30 18:44:39.508269 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.508239 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.508343 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.508298 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:39.508343 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:39.508328 5070 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 18:44:40.492473 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.492439 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" event={"ID":"7e87a8d3-748e-4674-9da3-e54a3ce24b87","Type":"ContainerStarted","Data":"7749fb5380304606938ab2aedb3bfe33108fa0de06f49454b439eae754c1f93f"} Oct 30 18:44:40.492664 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.492491 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" event={"ID":"7e87a8d3-748e-4674-9da3-e54a3ce24b87","Type":"ContainerStarted","Data":"6cc5b0e96b699b02b2f925e776a11d9d23bde8ac140f5999cace3bd36d8f35e9"} Oct 30 18:44:40.493979 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.493955 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" event={"ID":"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf","Type":"ContainerStarted","Data":"5b5227f336ffa11fe295d740632241cc3f23a5641477a86bf19e70a470bc8662"} Oct 30 18:44:40.494328 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.493995 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" event={"ID":"7d625d9d-56f7-4149-b9e2-5a8b5f114aaf","Type":"ContainerStarted","Data":"1def82acab7e1584988e17318ceab69019c59a7c7c5a6b0eff2b8aeaeedcf3de"} Oct 30 18:44:40.571519 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.571466 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podStartSLOduration=2.571444236 podStartE2EDuration="2.571444236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.56433624 +0000 UTC m=+797.679470161" watchObservedRunningTime="2026-10-30 18:44:40.571444236 +0000 UTC m=+797.686578152" Oct 30 18:44:40.612670 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.612623 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podStartSLOduration=2.612607487 podStartE2EDuration="2.612607487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:40.610794984 +0000 UTC m=+797.725928899" watchObservedRunningTime="2026-10-30 18:44:40.612607487 +0000 UTC m=+797.727741405" Oct 30 18:44:40.670790 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:40.670711 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:41.135164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.132591 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:41.135164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.135048 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.135164 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.135102 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:41.175000 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.174968 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:41.176228 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.176208 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:41.176289 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.176248 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:41.317169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.314833 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq"] Oct 30 18:44:41.371169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.370447 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq"] Oct 30 18:44:41.371169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.370496 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.390457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.390258 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-93f8ac45085c\"" Oct 30 18:44:41.390457 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.390376 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\"" Oct 30 18:44:41.403012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.402996 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\"" Oct 30 18:44:41.403137 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.403008 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-jbnzt\"" Oct 30 18:44:41.403207 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.403075 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\"" Oct 30 18:44:41.403207 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.403104 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\"" Oct 30 18:44:41.464172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.463906 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.464172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.464013 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.464172 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.464059 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6hf2\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.465839 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.464863 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.565837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.565656 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.565837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.565699 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6hf2\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.565837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.565727 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.565837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.565795 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.566508 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.566482 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.566984 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.566959 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.573740 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.573717 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.597771 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.597741 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6hf2\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2\") pod \"payload-processing-external-model-d66597b7c-bc8tq\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:41.685031 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:41.684948 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:42.064424 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.063516 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq"] Oct 30 18:44:42.070596 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:44:42.070419 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce91056_34da_419a_96b2_507e31035157.slice/crio-cbbc924d8711ed192432abd572e943b86b12e4bacea2c060f4d75dc5297d35a3 WatchSource:0}: Error finding container cbbc924d8711ed192432abd572e943b86b12e4bacea2c060f4d75dc5297d35a3: Status 404 returned error can't find the container with id cbbc924d8711ed192432abd572e943b86b12e4bacea2c060f4d75dc5297d35a3 Oct 30 18:44:42.138192 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.134583 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.138192 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.134640 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:42.175778 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.175751 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:42.175913 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.175790 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:42.506816 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.506772 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" event={"ID":"9ce91056-34da-419a-96b2-507e31035157","Type":"ContainerStarted","Data":"e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e"} Oct 30 18:44:42.507004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.506835 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" event={"ID":"9ce91056-34da-419a-96b2-507e31035157","Type":"ContainerStarted","Data":"cbbc924d8711ed192432abd572e943b86b12e4bacea2c060f4d75dc5297d35a3"} Oct 30 18:44:42.545373 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.541835 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:42.583171 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:42.579451 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" podStartSLOduration=1.579431703 podStartE2EDuration="1.579431703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:42.552604466 +0000 UTC m=+799.667738385" watchObservedRunningTime="2026-10-30 18:44:42.579431703 +0000 UTC m=+799.694565622" Oct 30 18:44:43.135167 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:43.134557 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.135167 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:43.134623 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:43.175118 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:43.175082 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:43.175279 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:43.175128 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:43.513033 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:43.512963 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:44.133168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:44.132606 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:44.133168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:44.132670 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:44.175721 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:44.175683 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:44.175868 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:44.175731 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:44.449977 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:44.449840 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:45.142167 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:45.141606 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:45.142167 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:45.141672 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:45.175453 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:45.175415 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:45.175630 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:45.175466 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:46.131216 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:46.131176 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:46.131418 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:46.131237 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:46.175438 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:46.175403 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:46.175978 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:46.175465 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:46.280974 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:46.280939 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:47.131603 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.131547 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-tzpc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:44:47.131880 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.131848 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" podUID="7d625d9d-56f7-4149-b9e2-5a8b5f114aaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:44:47.180206 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.178354 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 18:44:47.180206 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.178408 5070 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 18:44:47.964343 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.964298 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qqbk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:44:47.964633 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.964586 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qqbk5" podUID="f32b9d16-e1d8-4d54-9434-261254241d83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:44:47.988831 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:47.988702 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:48.018309 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.018272 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-6njrd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:44:48.019813 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.019776 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-6njrd" podUID="72ebd198-f18a-4a53-a8d8-8ac880573147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:44:48.631208 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.631176 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:48.632466 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.632406 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:48.633460 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.633439 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" Oct 30 18:44:48.848708 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.848677 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:48.849410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.849387 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:48.850622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:48.850601 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-tzpc7" Oct 30 18:44:49.646584 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:49.646546 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:51.444692 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:51.444659 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:53.142613 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:53.142576 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:54.549229 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:54.549197 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:44:54.839795 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:54.839695 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:56.538386 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:56.538355 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:58.139183 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:58.139101 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:59.830989 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:44:59.830951 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:00.197082 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.197046 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62"] Oct 30 18:45:00.201174 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.201135 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z"] Oct 30 18:45:00.201300 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.201243 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.205088 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.205070 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.225754 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.225728 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62"] Oct 30 18:45:00.236308 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.236284 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z"] Oct 30 18:45:00.275624 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.275589 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drszl\" (UniqueName: \"kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl\") pod \"maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.275800 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.275779 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.275856 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.275837 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.275934 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.275910 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7nnb\" (UniqueName: \"kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb\") pod \"maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.302268 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.302244 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:45:00.376683 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.376651 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.376821 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.376701 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.376873 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.376821 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7nnb\" (UniqueName: \"kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb\") pod \"maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.376930 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.376915 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drszl\" (UniqueName: \"kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl\") pod \"maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.377372 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.377354 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.377411 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.377362 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.400709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.400685 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7nnb\" (UniqueName: \"kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb\") pod \"maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.400950 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.400933 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drszl\" (UniqueName: \"kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl\") pod \"maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.517108 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.517029 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:00.524612 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.524584 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:00.538205 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.538140 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:00.695464 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.695432 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:00.845975 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.845941 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62"] Oct 30 18:45:00.847949 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:45:00.847918 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b4fdfa_50cd_4ff0_be09_52b8105b1a7f.slice/crio-2cf8f3a865d866636ff513aa752e7f24cc396aa272fe97ae2a11d269f6a4a431 WatchSource:0}: Error finding container 2cf8f3a865d866636ff513aa752e7f24cc396aa272fe97ae2a11d269f6a4a431: Status 404 returned error can't find the container with id 2cf8f3a865d866636ff513aa752e7f24cc396aa272fe97ae2a11d269f6a4a431 Oct 30 18:45:00.875497 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:00.875477 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z"] Oct 30 18:45:00.890289 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:45:00.890253 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5464249e_65ef_4dd4_b4a4_5e5d09c263c6.slice/crio-3cd9d2efcfb3697a387dfc4bf9ffc8982e206b8df31624d9d87b517851817454 WatchSource:0}: Error finding container 3cd9d2efcfb3697a387dfc4bf9ffc8982e206b8df31624d9d87b517851817454: Status 404 returned error can't find the container with id 3cd9d2efcfb3697a387dfc4bf9ffc8982e206b8df31624d9d87b517851817454 Oct 30 18:45:01.667948 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.667898 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" event={"ID":"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f","Type":"ContainerStarted","Data":"3c8c92ae60d38e37588c896725c5ab3fb1d6042bf9a4f75c9efd6fd5577cf8e1"} Oct 30 18:45:01.668188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.667996 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" event={"ID":"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f","Type":"ContainerStarted","Data":"2cf8f3a865d866636ff513aa752e7f24cc396aa272fe97ae2a11d269f6a4a431"} Oct 30 18:45:01.669481 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.669457 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerStarted","Data":"e16a8aa4737a6111a5108d5600046345a1de6575a12434f7a89c73b71087eb01"} Oct 30 18:45:01.669588 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.669500 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerStarted","Data":"3cd9d2efcfb3697a387dfc4bf9ffc8982e206b8df31624d9d87b517851817454"} Oct 30 18:45:01.711311 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.711263 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" podStartSLOduration=1.7112501249999998 podStartE2EDuration="1.711250125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:01.709412431 +0000 UTC m=+818.824546348" watchObservedRunningTime="2026-10-30 18:45:01.711250125 +0000 UTC m=+818.826384040" Oct 30 18:45:01.745321 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.745287 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:01.753803 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:01.753754 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" podStartSLOduration=1.7537402420000001 podStartE2EDuration="1.753740242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:01.750610094 +0000 UTC m=+818.865744010" watchObservedRunningTime="2026-10-30 18:45:01.753740242 +0000 UTC m=+818.868874158" Oct 30 18:45:02.677286 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:02.677250 5070 generic.go:356] "Generic (PLEG): container finished" podID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerID="e16a8aa4737a6111a5108d5600046345a1de6575a12434f7a89c73b71087eb01" exitCode=60 Oct 30 18:45:02.677737 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:02.677345 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerDied","Data":"e16a8aa4737a6111a5108d5600046345a1de6575a12434f7a89c73b71087eb01"} Oct 30 18:45:02.677737 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:02.677600 5070 scope.go:122] "RemoveContainer" containerID="e16a8aa4737a6111a5108d5600046345a1de6575a12434f7a89c73b71087eb01" Oct 30 18:45:02.679351 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:02.679334 5070 generic.go:356] "Generic (PLEG): container finished" podID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerID="3c8c92ae60d38e37588c896725c5ab3fb1d6042bf9a4f75c9efd6fd5577cf8e1" exitCode=60 Oct 30 18:45:02.679423 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:02.679402 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" event={"ID":"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f","Type":"ContainerDied","Data":"3c8c92ae60d38e37588c896725c5ab3fb1d6042bf9a4f75c9efd6fd5577cf8e1"} Oct 30 18:45:02.679678 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:02.679658 5070 scope.go:122] "RemoveContainer" containerID="3c8c92ae60d38e37588c896725c5ab3fb1d6042bf9a4f75c9efd6fd5577cf8e1" Oct 30 18:45:03.557830 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.557786 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:03.688478 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.688435 5070 generic.go:356] "Generic (PLEG): container finished" podID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerID="dd53f17edcb1cc8255ec52b03da4d91959d2e89d1ac205fa901f5eaebfe4d5eb" exitCode=60 Oct 30 18:45:03.688923 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.688524 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" event={"ID":"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f","Type":"ContainerDied","Data":"dd53f17edcb1cc8255ec52b03da4d91959d2e89d1ac205fa901f5eaebfe4d5eb"} Oct 30 18:45:03.688923 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.688587 5070 scope.go:122] "RemoveContainer" containerID="3c8c92ae60d38e37588c896725c5ab3fb1d6042bf9a4f75c9efd6fd5577cf8e1" Oct 30 18:45:03.688923 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.688839 5070 scope.go:122] "RemoveContainer" containerID="dd53f17edcb1cc8255ec52b03da4d91959d2e89d1ac205fa901f5eaebfe4d5eb" Oct 30 18:45:03.689179 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:03.689156 5070 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62_odh-ai-gateway-infra(f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" Oct 30 18:45:03.690698 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.690676 5070 generic.go:356] "Generic (PLEG): container finished" podID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerID="d99d6fd26e53eb30d154864289f3522e51d6553a63bb5dffc79f34c729435dfa" exitCode=60 Oct 30 18:45:03.690805 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.690708 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerDied","Data":"d99d6fd26e53eb30d154864289f3522e51d6553a63bb5dffc79f34c729435dfa"} Oct 30 18:45:03.691012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.690996 5070 scope.go:122] "RemoveContainer" containerID="d99d6fd26e53eb30d154864289f3522e51d6553a63bb5dffc79f34c729435dfa" Oct 30 18:45:03.691278 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:03.691258 5070 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z_odh-ai-gateway-infra(5464249e-65ef-4dd4-b4a4-5e5d09c263c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" Oct 30 18:45:03.718804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:03.718777 5070 scope.go:122] "RemoveContainer" containerID="e16a8aa4737a6111a5108d5600046345a1de6575a12434f7a89c73b71087eb01" Oct 30 18:45:03.833087 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:03.833008 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:45:04.698433 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:04.698405 5070 scope.go:122] "RemoveContainer" containerID="d99d6fd26e53eb30d154864289f3522e51d6553a63bb5dffc79f34c729435dfa" Oct 30 18:45:04.698870 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:04.698708 5070 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z_odh-ai-gateway-infra(5464249e-65ef-4dd4-b4a4-5e5d09c263c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" Oct 30 18:45:04.699916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:04.699899 5070 scope.go:122] "RemoveContainer" containerID="dd53f17edcb1cc8255ec52b03da4d91959d2e89d1ac205fa901f5eaebfe4d5eb" Oct 30 18:45:04.700119 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:04.700103 5070 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62_odh-ai-gateway-infra(f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" Oct 30 18:45:05.243628 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:05.243590 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:06.839790 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:06.839758 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:08.676609 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:08.676253 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:10.585169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:10.584168 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:12.423485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:12.423449 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:14.049909 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:14.049875 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:14.660818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:14.660756 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq"] Oct 30 18:45:15.751391 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:15.751351 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:16.026072 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:16.025992 5070 scope.go:122] "RemoveContainer" containerID="dd53f17edcb1cc8255ec52b03da4d91959d2e89d1ac205fa901f5eaebfe4d5eb" Oct 30 18:45:16.868685 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:16.868658 5070 generic.go:356] "Generic (PLEG): container finished" podID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerID="da77937d72746011647de434403b3c24692c41ff05f457be529118f01c470f77" exitCode=60 Oct 30 18:45:16.869079 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:16.868733 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" event={"ID":"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f","Type":"ContainerDied","Data":"da77937d72746011647de434403b3c24692c41ff05f457be529118f01c470f77"} Oct 30 18:45:16.869079 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:16.868810 5070 scope.go:122] "RemoveContainer" containerID="dd53f17edcb1cc8255ec52b03da4d91959d2e89d1ac205fa901f5eaebfe4d5eb" Oct 30 18:45:16.869266 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:16.869229 5070 scope.go:122] "RemoveContainer" containerID="da77937d72746011647de434403b3c24692c41ff05f457be529118f01c470f77" Oct 30 18:45:16.869550 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:16.869524 5070 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62_odh-ai-gateway-infra(f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" Oct 30 18:45:17.629705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:17.629653 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:17.785960 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:17.785911 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:18.032227 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:18.026978 5070 scope.go:122] "RemoveContainer" containerID="d99d6fd26e53eb30d154864289f3522e51d6553a63bb5dffc79f34c729435dfa" Oct 30 18:45:18.175462 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:18.172198 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62"] Oct 30 18:45:18.984243 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:18.974423 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:19.196174 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.194463 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:19.296224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.293364 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca\") pod \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " Oct 30 18:45:19.296224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.293505 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb\" (UniqueName: \"kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb\") pod \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\" (UID: \"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f\") " Oct 30 18:45:19.312175 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.309710 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca" pod "f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" (UID: "f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:19.400394 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.400346 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-maas-api-service-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:19.461168 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.458332 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb" pod "f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" (UID: "f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f"). InnerVolumeSpecName "kube-api-access-p7nnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:19.492026 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.490103 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerStarted","Data":"823aaa6be2f93756c8a82ec9005e8dae84a972d2409fb39215e5cb359b7be45c"} Oct 30 18:45:19.506042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.506009 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" event={"ID":"f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f","Type":"ContainerDied","Data":"2cf8f3a865d866636ff513aa752e7f24cc396aa272fe97ae2a11d269f6a4a431"} Oct 30 18:45:19.506301 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.506267 5070 scope.go:122] "RemoveContainer" containerID="da77937d72746011647de434403b3c24692c41ff05f457be529118f01c470f77" Oct 30 18:45:19.517076 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.508223 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62" Oct 30 18:45:19.530369 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.530348 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7nnb\" (UniqueName: \"kubernetes.io/projected/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f-kube-api-access-p7nnb\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:19.599894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.599868 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:19.667745 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.667627 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" podUID="9ce91056-34da-419a-96b2-507e31035157" containerName="payload-processing" containerID="cri-o://e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e" gracePeriod=25 Oct 30 18:45:19.710360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.685524 5070 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-cvhp5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 18:45:19.710360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.685576 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-cvhp5" podUID="7e87a8d3-748e-4674-9da3-e54a3ce24b87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 18:45:19.799176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.797842 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62"] Oct 30 18:45:19.805515 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:19.805483 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f6f7d5-29889765-xlt62"] Oct 30 18:45:20.138499 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.138477 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:45:20.255481 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.255449 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials\") pod \"9ce91056-34da-419a-96b2-507e31035157\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " Oct 30 18:45:20.255978 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.255499 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume\") pod \"9ce91056-34da-419a-96b2-507e31035157\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " Oct 30 18:45:20.255978 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.255555 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2\") pod \"9ce91056-34da-419a-96b2-507e31035157\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " Oct 30 18:45:20.255978 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.255626 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume\") pod \"9ce91056-34da-419a-96b2-507e31035157\" (UID: \"9ce91056-34da-419a-96b2-507e31035157\") " Oct 30 18:45:20.256123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.255974 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume" pod "9ce91056-34da-419a-96b2-507e31035157" (UID: "9ce91056-34da-419a-96b2-507e31035157"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:20.256123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.256044 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume" pod "9ce91056-34da-419a-96b2-507e31035157" (UID: "9ce91056-34da-419a-96b2-507e31035157"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:20.256220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.256163 5070 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-routing-overlay-volume\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:20.256220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.256177 5070 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce91056-34da-419a-96b2-507e31035157-plugins-config-volume\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:20.257827 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.257810 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials" pod "9ce91056-34da-419a-96b2-507e31035157" (UID: "9ce91056-34da-419a-96b2-507e31035157"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:20.257871 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.257821 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2" pod "9ce91056-34da-419a-96b2-507e31035157" (UID: "9ce91056-34da-419a-96b2-507e31035157"). InnerVolumeSpecName "kube-api-access-f6hf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:20.357653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.357627 5070 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-provider-credentials\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:20.357653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.357653 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6hf2\" (UniqueName: \"kubernetes.io/projected/9ce91056-34da-419a-96b2-507e31035157-kube-api-access-f6hf2\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:20.513644 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.513613 5070 generic.go:356] "Generic (PLEG): container finished" podID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerID="823aaa6be2f93756c8a82ec9005e8dae84a972d2409fb39215e5cb359b7be45c" exitCode=60 Oct 30 18:45:20.513793 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.513683 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerDied","Data":"823aaa6be2f93756c8a82ec9005e8dae84a972d2409fb39215e5cb359b7be45c"} Oct 30 18:45:20.513793 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.513739 5070 scope.go:122] "RemoveContainer" containerID="d99d6fd26e53eb30d154864289f3522e51d6553a63bb5dffc79f34c729435dfa" Oct 30 18:45:20.514050 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.514027 5070 scope.go:122] "RemoveContainer" containerID="823aaa6be2f93756c8a82ec9005e8dae84a972d2409fb39215e5cb359b7be45c" Oct 30 18:45:20.514356 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:20.514335 5070 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z_odh-ai-gateway-infra(5464249e-65ef-4dd4-b4a4-5e5d09c263c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" Oct 30 18:45:20.516117 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.516098 5070 generic.go:356] "Generic (PLEG): container finished" podID="9ce91056-34da-419a-96b2-507e31035157" containerID="e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e" exitCode=0 Oct 30 18:45:20.516219 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.516135 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" Oct 30 18:45:20.516219 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.516199 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" event={"ID":"9ce91056-34da-419a-96b2-507e31035157","Type":"ContainerDied","Data":"e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e"} Oct 30 18:45:20.516305 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.516230 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq" event={"ID":"9ce91056-34da-419a-96b2-507e31035157","Type":"ContainerDied","Data":"cbbc924d8711ed192432abd572e943b86b12e4bacea2c060f4d75dc5297d35a3"} Oct 30 18:45:20.525787 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.525771 5070 scope.go:122] "RemoveContainer" containerID="e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e" Oct 30 18:45:20.533724 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.533709 5070 scope.go:122] "RemoveContainer" containerID="e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e" Oct 30 18:45:20.533964 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:20.533947 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e\": container with ID starting with e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e not found: ID does not exist" containerID="e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e" Oct 30 18:45:20.534025 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.533977 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e"} err="failed to get container status \"e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e\": rpc error: code = NotFound desc = could not find container \"e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e\": container with ID starting with e276c0d80f2de9b64d4a221be93beb36acd7cd5e6bd9120aa05e4d23d71e065e not found: ID does not exist" Oct 30 18:45:20.565870 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.565846 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z"] Oct 30 18:45:20.600063 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.600033 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq"] Oct 30 18:45:20.602197 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:20.602167 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-d66597b7c-bc8tq"] Oct 30 18:45:21.032666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.032637 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ce91056-34da-419a-96b2-507e31035157" path="/var/lib/kubelet/pods/9ce91056-34da-419a-96b2-507e31035157/volumes" Oct 30 18:45:21.033042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.033030 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" path="/var/lib/kubelet/pods/f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f/volumes" Oct 30 18:45:21.254360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.254326 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:21.670421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.670397 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:21.773276 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.773245 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl\" (UniqueName: \"kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl\") pod \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " Oct 30 18:45:21.773413 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.773310 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca\") pod \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\" (UID: \"5464249e-65ef-4dd4-b4a4-5e5d09c263c6\") " Oct 30 18:45:21.773738 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.773704 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca" pod "5464249e-65ef-4dd4-b4a4-5e5d09c263c6" (UID: "5464249e-65ef-4dd4-b4a4-5e5d09c263c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:21.775483 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.775438 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl" pod "5464249e-65ef-4dd4-b4a4-5e5d09c263c6" (UID: "5464249e-65ef-4dd4-b4a4-5e5d09c263c6"). InnerVolumeSpecName "kube-api-access-drszl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:21.874692 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.874656 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-maas-api-service-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:21.874692 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.874684 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drszl\" (UniqueName: \"kubernetes.io/projected/5464249e-65ef-4dd4-b4a4-5e5d09c263c6-kube-api-access-drszl\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:45:21.925532 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:21.925508 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:22.531245 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:22.531212 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" Oct 30 18:45:22.531416 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:22.531247 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z" event={"ID":"5464249e-65ef-4dd4-b4a4-5e5d09c263c6","Type":"ContainerDied","Data":"3cd9d2efcfb3697a387dfc4bf9ffc8982e206b8df31624d9d87b517851817454"} Oct 30 18:45:22.531416 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:22.531332 5070 scope.go:122] "RemoveContainer" containerID="823aaa6be2f93756c8a82ec9005e8dae84a972d2409fb39215e5cb359b7be45c" Oct 30 18:45:22.589681 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:22.589651 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z"] Oct 30 18:45:22.598108 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:22.598087 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a759a1-29889765-7ls6z"] Oct 30 18:45:22.945118 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:22.945084 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:23.033568 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:23.033532 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" path="/var/lib/kubelet/pods/5464249e-65ef-4dd4-b4a4-5e5d09c263c6/volumes" Oct 30 18:45:24.723003 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:24.722973 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:26.266136 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:26.266091 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:27.856489 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:27.856457 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:29.599493 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:29.599461 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:31.253006 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:31.252966 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:32.856439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:32.856403 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:33.832302 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:45:33.832264 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:45:34.317952 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:34.317925 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:34.489311 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:34.489277 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:34.675895 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:34.675848 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:36.432715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:36.432671 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:37.964006 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:37.963974 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:39.745485 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:39.745433 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:41.346040 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:41.345857 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:43.064727 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:43.064683 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:44.736225 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:44.736181 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:45.934608 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:45.934571 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:46.601426 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:46.601382 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:47.141580 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:47.141542 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:47.369908 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:47.369877 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:47.545876 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:47.545793 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:48.551842 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:48.551797 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:50.478587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:50.478545 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:50.846868 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:50.846761 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r"] Oct 30 18:45:51.011890 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:51.011856 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:52.411736 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:52.411698 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:54.153420 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:54.153385 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:55.813355 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:55.813319 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:56.981116 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:56.981080 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:57.138338 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:57.138297 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:57.428773 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:57.428739 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:59.063757 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:45:59.063716 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:00.762937 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:00.762900 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:02.602914 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:02.602872 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:03.526557 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:03.526529 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:03.834113 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:03.834023 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:46:04.271860 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:04.271830 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:05.720177 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720123 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh"] Oct 30 18:46:05.720639 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720617 5070 cpu_manager.go:414] "containerMap: removing container" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.720639 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720634 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720649 5070 cpu_manager.go:414] "containerMap: removing container" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720655 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720669 5070 cpu_manager.go:414] "containerMap: removing container" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720674 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720687 5070 cpu_manager.go:414] "containerMap: removing container" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720692 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720704 5070 cpu_manager.go:414] "containerMap: removing container" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720709 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720717 5070 cpu_manager.go:414] "containerMap: removing container" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720721 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720732 5070 cpu_manager.go:414] "containerMap: removing container" podUID="9ce91056-34da-419a-96b2-507e31035157" containerName="payload-processing" Oct 30 18:46:05.720784 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720736 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ce91056-34da-419a-96b2-507e31035157" containerName="payload-processing" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720825 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ce91056-34da-419a-96b2-507e31035157" containerName="payload-processing" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720843 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720853 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720862 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720872 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2b4fdfa-50cd-4ff0-be09-52b8105b1a7f" containerName="cleanup" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720882 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.721221 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.720890 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="5464249e-65ef-4dd4-b4a4-5e5d09c263c6" containerName="cleanup" Oct 30 18:46:05.724638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.724615 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.746110 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.746082 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh"] Oct 30 18:46:05.755602 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.755573 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:05.755776 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.755762 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:05.877527 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.877488 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" podUID="c5cfe666-a504-4b72-8fa2-ea8403695758" containerName="main" containerID="cri-o://b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93" gracePeriod=45 Oct 30 18:46:05.897533 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.897504 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.897662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.897642 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph774\" (UniqueName: \"kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.897709 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.897696 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.998587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.998560 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph774\" (UniqueName: \"kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.998718 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.998623 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.998818 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.998720 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:05.999307 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:05.999284 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:06.000947 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.000921 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:06.023632 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.023577 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph774\" (UniqueName: \"kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774\") pod \"maas-api-revoke-sub-77bbf9c4-2knjh\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:06.042591 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.042560 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:06.148810 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.148753 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:06.154683 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.154659 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:46:06.301887 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.301812 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs\" (UniqueName: \"kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs\") pod \"c5cfe666-a504-4b72-8fa2-ea8403695758\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " Oct 30 18:46:06.301887 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.301869 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache\") pod \"c5cfe666-a504-4b72-8fa2-ea8403695758\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " Oct 30 18:46:06.302066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.301922 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir\") pod \"c5cfe666-a504-4b72-8fa2-ea8403695758\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " Oct 30 18:46:06.302066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.301967 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm\") pod \"c5cfe666-a504-4b72-8fa2-ea8403695758\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " Oct 30 18:46:06.302066 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302059 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home\") pod \"c5cfe666-a504-4b72-8fa2-ea8403695758\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " Oct 30 18:46:06.302215 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302102 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv\" (UniqueName: \"kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv\") pod \"c5cfe666-a504-4b72-8fa2-ea8403695758\" (UID: \"c5cfe666-a504-4b72-8fa2-ea8403695758\") " Oct 30 18:46:06.302215 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302191 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache" pod "c5cfe666-a504-4b72-8fa2-ea8403695758" (UID: "c5cfe666-a504-4b72-8fa2-ea8403695758"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.302331 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302200 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir" pod "c5cfe666-a504-4b72-8fa2-ea8403695758" (UID: "c5cfe666-a504-4b72-8fa2-ea8403695758"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.302385 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302354 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home" pod "c5cfe666-a504-4b72-8fa2-ea8403695758" (UID: "c5cfe666-a504-4b72-8fa2-ea8403695758"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.302561 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302538 5070 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-home\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.302561 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302560 5070 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-model-cache\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.302762 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.302581 5070 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-tmp-dir\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.306653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.306626 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs" pod "c5cfe666-a504-4b72-8fa2-ea8403695758" (UID: "c5cfe666-a504-4b72-8fa2-ea8403695758"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:06.306771 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.306673 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv" pod "c5cfe666-a504-4b72-8fa2-ea8403695758" (UID: "c5cfe666-a504-4b72-8fa2-ea8403695758"). InnerVolumeSpecName "kube-api-access-kd7xv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:06.306812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.306782 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm" pod "c5cfe666-a504-4b72-8fa2-ea8403695758" (UID: "c5cfe666-a504-4b72-8fa2-ea8403695758"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:06.363484 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.363458 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh"] Oct 30 18:46:06.365032 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:46:06.365005 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c0be8c5_bdf6_4844_8bf7_65fa97f4d5ae.slice/crio-cd21d3e6e89a476a74f66e074b215dbbe274940d65c5c6bc0f4c912568382254 WatchSource:0}: Error finding container cd21d3e6e89a476a74f66e074b215dbbe274940d65c5c6bc0f4c912568382254: Status 404 returned error can't find the container with id cd21d3e6e89a476a74f66e074b215dbbe274940d65c5c6bc0f4c912568382254 Oct 30 18:46:06.403261 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.403242 5070 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5cfe666-a504-4b72-8fa2-ea8403695758-dshm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.403337 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.403262 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd7xv\" (UniqueName: \"kubernetes.io/projected/c5cfe666-a504-4b72-8fa2-ea8403695758-kube-api-access-kd7xv\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.403337 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.403273 5070 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5cfe666-a504-4b72-8fa2-ea8403695758-tls-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:06.970588 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.970539 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" event={"ID":"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae","Type":"ContainerStarted","Data":"308800b9f55131bdc79fdc6ad5f7d48a6d49cb0c7a8a9d50ed500a8e5ca1bf31"} Oct 30 18:46:06.971031 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.970609 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" event={"ID":"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae","Type":"ContainerStarted","Data":"cd21d3e6e89a476a74f66e074b215dbbe274940d65c5c6bc0f4c912568382254"} Oct 30 18:46:06.971925 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.971900 5070 generic.go:356] "Generic (PLEG): container finished" podID="c5cfe666-a504-4b72-8fa2-ea8403695758" containerID="b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93" exitCode=0 Oct 30 18:46:06.972052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.971944 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" Oct 30 18:46:06.972052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.971971 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" event={"ID":"c5cfe666-a504-4b72-8fa2-ea8403695758","Type":"ContainerDied","Data":"b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93"} Oct 30 18:46:06.972052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.972004 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r" event={"ID":"c5cfe666-a504-4b72-8fa2-ea8403695758","Type":"ContainerDied","Data":"7f395ebecf3fed5b72f0f99d9cd1707a6ff48f674957182367217e0b5cf361db"} Oct 30 18:46:06.972052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.972018 5070 scope.go:122] "RemoveContainer" containerID="b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93" Oct 30 18:46:06.982311 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.982292 5070 scope.go:122] "RemoveContainer" containerID="b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93" Oct 30 18:46:06.982569 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:06.982552 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93\": container with ID starting with b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93 not found: ID does not exist" containerID="b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93" Oct 30 18:46:06.982612 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:06.982578 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93"} err="failed to get container status \"b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93\": rpc error: code = NotFound desc = could not find container \"b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93\": container with ID starting with b5a893e7cce38ea1301d6b704c8be2f598095a0ffd3f349bd87fb239d1210c93 not found: ID does not exist" Oct 30 18:46:07.013655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:07.013614 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" podStartSLOduration=2.013602819 podStartE2EDuration="2.013602819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:07.013063914 +0000 UTC m=+884.128197830" watchObservedRunningTime="2026-10-30 18:46:07.013602819 +0000 UTC m=+884.128736748" Oct 30 18:46:07.051822 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:07.051778 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r"] Oct 30 18:46:07.056167 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:07.056132 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-a47385/simulator-w4-a47385-kserve-56867dbd67-8ms9r"] Oct 30 18:46:07.871474 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:07.871394 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:07.980133 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:07.980099 5070 generic.go:356] "Generic (PLEG): container finished" podID="4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" containerID="308800b9f55131bdc79fdc6ad5f7d48a6d49cb0c7a8a9d50ed500a8e5ca1bf31" exitCode=0 Oct 30 18:46:07.980611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:07.980182 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" event={"ID":"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae","Type":"ContainerDied","Data":"308800b9f55131bdc79fdc6ad5f7d48a6d49cb0c7a8a9d50ed500a8e5ca1bf31"} Oct 30 18:46:08.575236 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:08.575177 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:09.032574 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.032544 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5cfe666-a504-4b72-8fa2-ea8403695758" path="/var/lib/kubelet/pods/c5cfe666-a504-4b72-8fa2-ea8403695758/volumes" Oct 30 18:46:09.135012 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.134990 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:09.235638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.235605 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca\") pod \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " Oct 30 18:46:09.235804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.235681 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774\" (UniqueName: \"kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774\") pod \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " Oct 30 18:46:09.235804 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.235737 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca\") pod \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\" (UID: \"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae\") " Oct 30 18:46:09.236125 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.236094 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca" pod "4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" (UID: "4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:09.237832 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.237783 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca" pod "4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" (UID: "4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:09.237832 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.237823 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774" pod "4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" (UID: "4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae"). InnerVolumeSpecName "kube-api-access-ph774". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:09.336733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.336699 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-xks-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:09.336733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.336728 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph774\" (UniqueName: \"kubernetes.io/projected/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-kube-api-access-ph774\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:09.336733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.336738 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae-maas-api-openshift-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:09.473385 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.473354 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:09.934649 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.934618 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt"] Oct 30 18:46:09.935094 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.935080 5070 cpu_manager.go:414] "containerMap: removing container" podUID="c5cfe666-a504-4b72-8fa2-ea8403695758" containerName="main" Oct 30 18:46:09.935094 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.935093 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5cfe666-a504-4b72-8fa2-ea8403695758" containerName="main" Oct 30 18:46:09.935297 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.935124 5070 cpu_manager.go:414] "containerMap: removing container" podUID="4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" containerName="revoke-keys" Oct 30 18:46:09.935297 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.935130 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" containerName="revoke-keys" Oct 30 18:46:09.935297 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.935243 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae" containerName="revoke-keys" Oct 30 18:46:09.935297 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.935256 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5cfe666-a504-4b72-8fa2-ea8403695758" containerName="main" Oct 30 18:46:09.938698 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.938676 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:09.958916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.958893 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-0ddc26\"/\"e2e-conflict-b-6c6935b2-kserve-self-signed-certs\"" Oct 30 18:46:09.975936 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.975908 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt"] Oct 30 18:46:09.996655 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.996626 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" event={"ID":"4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae","Type":"ContainerDied","Data":"cd21d3e6e89a476a74f66e074b215dbbe274940d65c5c6bc0f4c912568382254"} Oct 30 18:46:09.996748 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.996685 5070 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd21d3e6e89a476a74f66e074b215dbbe274940d65c5c6bc0f4c912568382254" Oct 30 18:46:09.996748 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:09.996643 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-77bbf9c4-2knjh" Oct 30 18:46:10.045248 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.045213 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.045660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.045264 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.045660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.045303 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.045660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.045409 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb7jl\" (UniqueName: \"kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.045660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.045553 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.045660 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.045649 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.065969 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.065934 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:10.146585 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.146549 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.146829 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.146621 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb7jl\" (UniqueName: \"kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.146829 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.146744 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.148318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.146888 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.148318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.146991 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.148318 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.147057 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.149667 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.149625 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.150523 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.150497 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.150653 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.147642 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.153185 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.153162 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.153312 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.153162 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.177756 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.177719 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb7jl\" (UniqueName: \"kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl\") pod \"e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.253895 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.253795 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:10.557010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:10.556984 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt"] Oct 30 18:46:10.557407 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:46:10.557382 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5fb80ae_c927_4f2d_8212_735a691e3830.slice/crio-6a1e62aed5d31a8c90d4cc75e83f3290afc967bad724bffae9a0c06643040c4a WatchSource:0}: Error finding container 6a1e62aed5d31a8c90d4cc75e83f3290afc967bad724bffae9a0c06643040c4a: Status 404 returned error can't find the container with id 6a1e62aed5d31a8c90d4cc75e83f3290afc967bad724bffae9a0c06643040c4a Oct 30 18:46:11.003567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:11.003532 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" event={"ID":"b5fb80ae-c927-4f2d-8212-735a691e3830","Type":"ContainerStarted","Data":"61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee"} Oct 30 18:46:11.003567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:11.003577 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" event={"ID":"b5fb80ae-c927-4f2d-8212-735a691e3830","Type":"ContainerStarted","Data":"6a1e62aed5d31a8c90d4cc75e83f3290afc967bad724bffae9a0c06643040c4a"} Oct 30 18:46:11.049375 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:11.049314 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" podStartSLOduration=2.049300762 podStartE2EDuration="2.049300762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:11.046509984 +0000 UTC m=+888.161643918" watchObservedRunningTime="2026-10-30 18:46:11.049300762 +0000 UTC m=+888.164434679" Oct 30 18:46:11.170504 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:11.170457 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:11.254099 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:11.254011 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:12.939349 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:12.939315 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:13.890233 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:13.890195 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:14.993836 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:14.993806 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:15.209853 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:15.209818 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:17.143198 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:17.143132 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:17.887125 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:17.887091 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:18.950263 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:18.950226 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:19.105640 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:19.105610 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:20.012299 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:20.012262 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:20.254428 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:20.254393 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:20.267518 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:20.267440 5070 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:20.738534 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:20.738496 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:21.086047 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:21.085975 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:21.336034 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:21.336000 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:21.520767 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:21.520733 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:21.717138 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:21.717107 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:22.048636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:22.048607 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:22.334719 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:22.334641 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:23.137280 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:23.137255 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:46:23.142170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:23.142118 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:46:23.164940 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:23.164798 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:46:23.169578 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:23.169558 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6tz8_47df06d1-8c08-47c2-9e1f-10636f6bf035/ovn-acl-logging/0.log" Oct 30 18:46:23.611720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:23.611636 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:24.723840 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:24.723807 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:24.908068 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:24.908011 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:25.304985 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:25.304945 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:26.896396 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:26.896361 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:28.307703 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:28.307670 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:28.528856 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:28.528822 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:28.890712 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:28.890675 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:29.391113 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.391082 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:29.588386 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.588349 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk"] Oct 30 18:46:29.595385 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.595354 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.611702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.611671 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk"] Oct 30 18:46:29.620859 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.620826 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:29.620982 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.620912 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:29.705253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.702894 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.705253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.703046 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.705253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.703101 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9ff9\" (UniqueName: \"kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.804708 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.804675 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.804857 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.804719 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9ff9\" (UniqueName: \"kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.804857 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.804776 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.805359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.805339 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.807089 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.807068 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.828360 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.828334 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9ff9\" (UniqueName: \"kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9\") pod \"maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:29.909415 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:29.909386 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:30.230338 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:30.230259 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk"] Oct 30 18:46:30.234083 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:46:30.234055 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7497cfa8_e00d_48cf_9cb0_2bd4825337d4.slice/crio-5dd38832d0f05bfb8f7c727a32d7eb9f3c34f792f03ab6331513b4bcdd076f3f WatchSource:0}: Error finding container 5dd38832d0f05bfb8f7c727a32d7eb9f3c34f792f03ab6331513b4bcdd076f3f: Status 404 returned error can't find the container with id 5dd38832d0f05bfb8f7c727a32d7eb9f3c34f792f03ab6331513b4bcdd076f3f Oct 30 18:46:31.001754 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:31.001721 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:31.160562 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:31.160535 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" event={"ID":"7497cfa8-e00d-48cf-9cb0-2bd4825337d4","Type":"ContainerStarted","Data":"a4ffac9b0c5ce6306fc16acba08e237ef3320b0905549b59703f3dde2a8a0e74"} Oct 30 18:46:31.160711 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:31.160579 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" event={"ID":"7497cfa8-e00d-48cf-9cb0-2bd4825337d4","Type":"ContainerStarted","Data":"5dd38832d0f05bfb8f7c727a32d7eb9f3c34f792f03ab6331513b4bcdd076f3f"} Oct 30 18:46:31.212313 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:31.212266 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" podStartSLOduration=2.212253668 podStartE2EDuration="2.212253668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:31.209204751 +0000 UTC m=+908.324338667" watchObservedRunningTime="2026-10-30 18:46:31.212253668 +0000 UTC m=+908.327387584" Oct 30 18:46:31.438875 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:31.438846 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:31.989471 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:31.989435 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:32.167316 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:32.167282 5070 generic.go:356] "Generic (PLEG): container finished" podID="7497cfa8-e00d-48cf-9cb0-2bd4825337d4" containerID="a4ffac9b0c5ce6306fc16acba08e237ef3320b0905549b59703f3dde2a8a0e74" exitCode=0 Oct 30 18:46:32.167693 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:32.167359 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" event={"ID":"7497cfa8-e00d-48cf-9cb0-2bd4825337d4","Type":"ContainerDied","Data":"a4ffac9b0c5ce6306fc16acba08e237ef3320b0905549b59703f3dde2a8a0e74"} Oct 30 18:46:32.230622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:32.230579 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:32.433879 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:32.433809 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:33.138339 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.138295 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:33.325611 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.325589 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:33.446511 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.446414 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca\") pod \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " Oct 30 18:46:33.446511 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.446503 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca\") pod \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " Oct 30 18:46:33.446720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.446564 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9\" (UniqueName: \"kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9\") pod \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\" (UID: \"7497cfa8-e00d-48cf-9cb0-2bd4825337d4\") " Oct 30 18:46:33.446794 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.446773 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca" pod "7497cfa8-e00d-48cf-9cb0-2bd4825337d4" (UID: "7497cfa8-e00d-48cf-9cb0-2bd4825337d4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:33.446918 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.446904 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-openshift-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:33.448587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.448568 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca" pod "7497cfa8-e00d-48cf-9cb0-2bd4825337d4" (UID: "7497cfa8-e00d-48cf-9cb0-2bd4825337d4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:33.448674 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.448639 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9" pod "7497cfa8-e00d-48cf-9cb0-2bd4825337d4" (UID: "7497cfa8-e00d-48cf-9cb0-2bd4825337d4"). InnerVolumeSpecName "kube-api-access-j9ff9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:33.547506 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.547474 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9ff9\" (UniqueName: \"kubernetes.io/projected/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-kube-api-access-j9ff9\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:33.547506 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:33.547501 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7497cfa8-e00d-48cf-9cb0-2bd4825337d4-maas-api-xks-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:33.833702 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:33.833618 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:46:34.180696 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.180669 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" Oct 30 18:46:34.180900 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.180704 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk" event={"ID":"7497cfa8-e00d-48cf-9cb0-2bd4825337d4","Type":"ContainerDied","Data":"5dd38832d0f05bfb8f7c727a32d7eb9f3c34f792f03ab6331513b4bcdd076f3f"} Oct 30 18:46:34.180900 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.180749 5070 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5dd38832d0f05bfb8f7c727a32d7eb9f3c34f792f03ab6331513b4bcdd076f3f" Oct 30 18:46:34.319097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.319064 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7"] Oct 30 18:46:34.324642 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.324618 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq"] Oct 30 18:46:34.468777 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.468687 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:34.637263 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:34.637229 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:35.168507 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:35.168472 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:35.354723 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:35.354690 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:35.820335 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:35.820301 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:37.139904 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:37.139866 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:37.854853 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:37.854822 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:38.398826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:38.398794 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt"] Oct 30 18:46:38.483565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:38.483520 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:38.998401 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:38.998366 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:39.824812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:39.824775 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:40.081042 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:40.080957 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:40.491111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:40.491077 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:40.672888 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:40.672850 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:40.814395 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:40.814320 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd"] Oct 30 18:46:40.814556 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:40.814532 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" podUID="702ca51e-a16f-441b-a169-b29fb5789b48" containerName="maas-api" containerID="cri-o://a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc" gracePeriod=30 Oct 30 18:46:40.912493 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:40.912456 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:41.091704 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.091673 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:46:41.220665 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.220632 5070 generic.go:356] "Generic (PLEG): container finished" podID="702ca51e-a16f-441b-a169-b29fb5789b48" containerID="a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc" exitCode=0 Oct 30 18:46:41.220826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.220673 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" Oct 30 18:46:41.220826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.220717 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" event={"ID":"702ca51e-a16f-441b-a169-b29fb5789b48","Type":"ContainerDied","Data":"a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc"} Oct 30 18:46:41.220826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.220771 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd" event={"ID":"702ca51e-a16f-441b-a169-b29fb5789b48","Type":"ContainerDied","Data":"8eed99b84285810d530de5ce8f54e775103abb279b18eeff4ad80275a71e6188"} Oct 30 18:46:41.220826 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.220796 5070 scope.go:122] "RemoveContainer" containerID="a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc" Oct 30 18:46:41.230047 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.230030 5070 scope.go:122] "RemoveContainer" containerID="a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc" Oct 30 18:46:41.230313 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:41.230294 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc\": container with ID starting with a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc not found: ID does not exist" containerID="a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc" Oct 30 18:46:41.230362 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.230321 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc"} err="failed to get container status \"a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc\": rpc error: code = NotFound desc = could not find container \"a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc\": container with ID starting with a60238125692df6a390afc7140dc983c41105d44091800a0a0e42338f459dcfc not found: ID does not exist" Oct 30 18:46:41.240449 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.240429 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls\") pod \"702ca51e-a16f-441b-a169-b29fb5789b48\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " Oct 30 18:46:41.240540 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.240475 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs\") pod \"702ca51e-a16f-441b-a169-b29fb5789b48\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " Oct 30 18:46:41.240540 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.240517 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca\") pod \"702ca51e-a16f-441b-a169-b29fb5789b48\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " Oct 30 18:46:41.240630 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.240613 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4\" (UniqueName: \"kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4\") pod \"702ca51e-a16f-441b-a169-b29fb5789b48\" (UID: \"702ca51e-a16f-441b-a169-b29fb5789b48\") " Oct 30 18:46:41.240951 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.240925 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca" pod "702ca51e-a16f-441b-a169-b29fb5789b48" (UID: "702ca51e-a16f-441b-a169-b29fb5789b48"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:41.241067 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.241037 5070 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/702ca51e-a16f-441b-a169-b29fb5789b48-openshift-service-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:41.242565 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.242546 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls" pod "702ca51e-a16f-441b-a169-b29fb5789b48" (UID: "702ca51e-a16f-441b-a169-b29fb5789b48"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:41.242662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.242639 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4" pod "702ca51e-a16f-441b-a169-b29fb5789b48" (UID: "702ca51e-a16f-441b-a169-b29fb5789b48"). InnerVolumeSpecName "kube-api-access-fgtc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:41.242662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.242654 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs" pod "702ca51e-a16f-441b-a169-b29fb5789b48" (UID: "702ca51e-a16f-441b-a169-b29fb5789b48"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:41.342421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.342302 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgtc4\" (UniqueName: \"kubernetes.io/projected/702ca51e-a16f-441b-a169-b29fb5789b48-kube-api-access-fgtc4\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:41.342421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.342345 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-maas-api-tls\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:41.342421 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.342360 5070 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/702ca51e-a16f-441b-a169-b29fb5789b48-metrics-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:41.588352 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.588319 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd"] Oct 30 18:46:41.590482 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.590459 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f6f7d5-6fb9984858-rgbhd"] Oct 30 18:46:41.595391 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.595325 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:41.812243 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:41.812209 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:42.249722 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:42.249693 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:43.032552 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.032520 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="702ca51e-a16f-441b-a169-b29fb5789b48" path="/var/lib/kubelet/pods/702ca51e-a16f-441b-a169-b29fb5789b48/volumes" Oct 30 18:46:43.179958 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.179923 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h"] Oct 30 18:46:43.180493 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.180478 5070 cpu_manager.go:414] "containerMap: removing container" podUID="702ca51e-a16f-441b-a169-b29fb5789b48" containerName="maas-api" Oct 30 18:46:43.180567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.180495 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="702ca51e-a16f-441b-a169-b29fb5789b48" containerName="maas-api" Oct 30 18:46:43.180567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.180508 5070 cpu_manager.go:414] "containerMap: removing container" podUID="7497cfa8-e00d-48cf-9cb0-2bd4825337d4" containerName="revoke-keys" Oct 30 18:46:43.180567 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.180515 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7497cfa8-e00d-48cf-9cb0-2bd4825337d4" containerName="revoke-keys" Oct 30 18:46:43.180669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.180619 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="7497cfa8-e00d-48cf-9cb0-2bd4825337d4" containerName="revoke-keys" Oct 30 18:46:43.180669 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.180636 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="702ca51e-a16f-441b-a169-b29fb5789b48" containerName="maas-api" Oct 30 18:46:43.185230 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.185212 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.213614 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.213586 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h"] Oct 30 18:46:43.222343 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.222323 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 18:46:43.222525 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.222508 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ktn4z\"" Oct 30 18:46:43.260116 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.260082 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.260116 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.260119 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.260536 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.260195 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dltv5\" (UniqueName: \"kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.361306 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.361219 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.361306 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.361262 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.361306 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.361298 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dltv5\" (UniqueName: \"kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.361852 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.361829 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.363614 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.363591 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.385872 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.385834 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dltv5\" (UniqueName: \"kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5\") pod \"maas-api-revoke-sub-8f0806aa-s2r8h\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.498629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.498600 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:43.566174 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.566124 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:43.801251 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:43.801224 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h"] Oct 30 18:46:43.801882 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:46:43.801859 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15911411_0b80_46ef_ac4b_1ff42527fcfb.slice/crio-20fb113be00fbe925da462163b38f0b2195666396015b1e4410097f6bd3a5aba WatchSource:0}: Error finding container 20fb113be00fbe925da462163b38f0b2195666396015b1e4410097f6bd3a5aba: Status 404 returned error can't find the container with id 20fb113be00fbe925da462163b38f0b2195666396015b1e4410097f6bd3a5aba Oct 30 18:46:44.219798 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.219761 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:44.241704 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.241672 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" event={"ID":"15911411-0b80-46ef-ac4b-1ff42527fcfb","Type":"ContainerStarted","Data":"87e11173fd2ec31ffd06e3e49ae387478e7d0a7b720f6d419b73826a9d5f86e8"} Oct 30 18:46:44.241859 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.241718 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" event={"ID":"15911411-0b80-46ef-ac4b-1ff42527fcfb","Type":"ContainerStarted","Data":"20fb113be00fbe925da462163b38f0b2195666396015b1e4410097f6bd3a5aba"} Oct 30 18:46:44.290781 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.290731 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" podStartSLOduration=1.290717034 podStartE2EDuration="1.290717034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:44.2902337 +0000 UTC m=+921.405367618" watchObservedRunningTime="2026-10-30 18:46:44.290717034 +0000 UTC m=+921.405850951" Oct 30 18:46:44.401598 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.401512 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:44.623799 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.623761 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:44.986141 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:44.986056 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2"] Oct 30 18:46:45.141476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:45.141424 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:45.248675 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:45.248579 5070 generic.go:356] "Generic (PLEG): container finished" podID="15911411-0b80-46ef-ac4b-1ff42527fcfb" containerID="87e11173fd2ec31ffd06e3e49ae387478e7d0a7b720f6d419b73826a9d5f86e8" exitCode=0 Oct 30 18:46:45.248675 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:45.248622 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" event={"ID":"15911411-0b80-46ef-ac4b-1ff42527fcfb","Type":"ContainerDied","Data":"87e11173fd2ec31ffd06e3e49ae387478e7d0a7b720f6d419b73826a9d5f86e8"} Oct 30 18:46:45.312176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:45.311556 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:45.485222 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:45.485191 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:46.407103 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.407081 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:46.497884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.497847 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca\") pod \"15911411-0b80-46ef-ac4b-1ff42527fcfb\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " Oct 30 18:46:46.498051 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.497892 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5\" (UniqueName: \"kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5\") pod \"15911411-0b80-46ef-ac4b-1ff42527fcfb\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " Oct 30 18:46:46.498051 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.497920 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca\") pod \"15911411-0b80-46ef-ac4b-1ff42527fcfb\" (UID: \"15911411-0b80-46ef-ac4b-1ff42527fcfb\") " Oct 30 18:46:46.498367 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.498344 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca" pod "15911411-0b80-46ef-ac4b-1ff42527fcfb" (UID: "15911411-0b80-46ef-ac4b-1ff42527fcfb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:46.500041 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.500012 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca" pod "15911411-0b80-46ef-ac4b-1ff42527fcfb" (UID: "15911411-0b80-46ef-ac4b-1ff42527fcfb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:46.500041 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.500035 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5" pod "15911411-0b80-46ef-ac4b-1ff42527fcfb" (UID: "15911411-0b80-46ef-ac4b-1ff42527fcfb"). InnerVolumeSpecName "kube-api-access-dltv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:46.599361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.599332 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-xks-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:46.599361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.599359 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dltv5\" (UniqueName: \"kubernetes.io/projected/15911411-0b80-46ef-ac4b-1ff42527fcfb-kube-api-access-dltv5\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:46.599577 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:46.599371 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/15911411-0b80-46ef-ac4b-1ff42527fcfb-maas-api-openshift-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:47.261398 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:47.261362 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" event={"ID":"15911411-0b80-46ef-ac4b-1ff42527fcfb","Type":"ContainerDied","Data":"20fb113be00fbe925da462163b38f0b2195666396015b1e4410097f6bd3a5aba"} Oct 30 18:46:47.261398 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:47.261381 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8f0806aa-s2r8h" Oct 30 18:46:47.261596 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:47.261415 5070 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20fb113be00fbe925da462163b38f0b2195666396015b1e4410097f6bd3a5aba" Oct 30 18:46:47.393681 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:47.393649 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:47.589610 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:47.589577 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:49.016711 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.016677 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:49.182985 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.182953 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:49.339882 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.339800 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" podUID="7f60b6bc-6a69-47e6-bb5b-545aec37fd45" containerName="main" containerID="cri-o://742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868" gracePeriod=45 Oct 30 18:46:49.344029 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.343844 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" podUID="0f01e1bb-39a4-4233-a132-6f5f2c024dac" containerName="main" containerID="cri-o://6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf" gracePeriod=45 Oct 30 18:46:49.620510 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.620482 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:46:49.642659 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.642635 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:46:49.732584 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732544 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache\") pod \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " Oct 30 18:46:49.732768 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732598 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm\") pod \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " Oct 30 18:46:49.732768 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732644 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir\") pod \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " Oct 30 18:46:49.732768 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732700 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home\") pod \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " Oct 30 18:46:49.732935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732765 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf\" (UniqueName: \"kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf\") pod \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " Oct 30 18:46:49.732935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732822 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm\") pod \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " Oct 30 18:46:49.732935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732838 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache" pod "7f60b6bc-6a69-47e6-bb5b-545aec37fd45" (UID: "7f60b6bc-6a69-47e6-bb5b-545aec37fd45"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.732935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732875 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs\" (UniqueName: \"kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs\") pod \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\" (UID: \"7f60b6bc-6a69-47e6-bb5b-545aec37fd45\") " Oct 30 18:46:49.732935 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732906 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir\") pod \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " Oct 30 18:46:49.733224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.732973 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp\" (UniqueName: \"kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp\") pod \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " Oct 30 18:46:49.733224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733004 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache\") pod \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " Oct 30 18:46:49.733224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733053 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home" pod "7f60b6bc-6a69-47e6-bb5b-545aec37fd45" (UID: "7f60b6bc-6a69-47e6-bb5b-545aec37fd45"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.733224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733089 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs\") pod \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " Oct 30 18:46:49.733224 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733131 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home\") pod \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\" (UID: \"0f01e1bb-39a4-4233-a132-6f5f2c024dac\") " Oct 30 18:46:49.733572 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733543 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir" pod "7f60b6bc-6a69-47e6-bb5b-545aec37fd45" (UID: "7f60b6bc-6a69-47e6-bb5b-545aec37fd45"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.733702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733630 5070 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-model-cache\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.733702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733651 5070 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tmp-dir\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.733702 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733664 5070 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-home\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.733887 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.733709 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir" pod "0f01e1bb-39a4-4233-a132-6f5f2c024dac" (UID: "0f01e1bb-39a4-4233-a132-6f5f2c024dac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.734630 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.734600 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home" pod "0f01e1bb-39a4-4233-a132-6f5f2c024dac" (UID: "0f01e1bb-39a4-4233-a132-6f5f2c024dac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.734735 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.734717 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache" pod "0f01e1bb-39a4-4233-a132-6f5f2c024dac" (UID: "0f01e1bb-39a4-4233-a132-6f5f2c024dac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.736500 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.736476 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs" pod "0f01e1bb-39a4-4233-a132-6f5f2c024dac" (UID: "0f01e1bb-39a4-4233-a132-6f5f2c024dac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:49.736595 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.736581 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp" pod "0f01e1bb-39a4-4233-a132-6f5f2c024dac" (UID: "0f01e1bb-39a4-4233-a132-6f5f2c024dac"). InnerVolumeSpecName "kube-api-access-g76zp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.736694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.736671 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm" pod "0f01e1bb-39a4-4233-a132-6f5f2c024dac" (UID: "0f01e1bb-39a4-4233-a132-6f5f2c024dac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.736803 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.736727 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm" pod "7f60b6bc-6a69-47e6-bb5b-545aec37fd45" (UID: "7f60b6bc-6a69-47e6-bb5b-545aec37fd45"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:49.737185 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.737128 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs" pod "7f60b6bc-6a69-47e6-bb5b-545aec37fd45" (UID: "7f60b6bc-6a69-47e6-bb5b-545aec37fd45"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:49.737933 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.737904 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf" pod "7f60b6bc-6a69-47e6-bb5b-545aec37fd45" (UID: "7f60b6bc-6a69-47e6-bb5b-545aec37fd45"). InnerVolumeSpecName "kube-api-access-mmtbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:49.834915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834879 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmtbf\" (UniqueName: \"kubernetes.io/projected/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-kube-api-access-mmtbf\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.834915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834913 5070 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-dshm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834925 5070 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-tls-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834933 5070 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tmp-dir\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834945 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g76zp\" (UniqueName: \"kubernetes.io/projected/0f01e1bb-39a4-4233-a132-6f5f2c024dac-kube-api-access-g76zp\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834957 5070 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-model-cache\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834971 5070 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f01e1bb-39a4-4233-a132-6f5f2c024dac-tls-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.834994 5070 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f01e1bb-39a4-4233-a132-6f5f2c024dac-home\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.835199 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.835002 5070 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f60b6bc-6a69-47e6-bb5b-545aec37fd45-dshm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:49.953020 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:49.952943 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:50.279369 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.279272 5070 generic.go:356] "Generic (PLEG): container finished" podID="0f01e1bb-39a4-4233-a132-6f5f2c024dac" containerID="6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf" exitCode=0 Oct 30 18:46:50.279369 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.279333 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" Oct 30 18:46:50.279865 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.279360 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" event={"ID":"0f01e1bb-39a4-4233-a132-6f5f2c024dac","Type":"ContainerDied","Data":"6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf"} Oct 30 18:46:50.279865 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.279410 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq" event={"ID":"0f01e1bb-39a4-4233-a132-6f5f2c024dac","Type":"ContainerDied","Data":"b9ee8cc5ee2aab26aa3baa79f10988089b384b0f0a91ac0a462d1e8fdecf373c"} Oct 30 18:46:50.279865 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.279430 5070 scope.go:122] "RemoveContainer" containerID="6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf" Oct 30 18:46:50.280760 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.280736 5070 generic.go:356] "Generic (PLEG): container finished" podID="7f60b6bc-6a69-47e6-bb5b-545aec37fd45" containerID="742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868" exitCode=0 Oct 30 18:46:50.280861 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.280790 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" Oct 30 18:46:50.280861 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.280805 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" event={"ID":"7f60b6bc-6a69-47e6-bb5b-545aec37fd45","Type":"ContainerDied","Data":"742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868"} Oct 30 18:46:50.280861 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.280831 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7" event={"ID":"7f60b6bc-6a69-47e6-bb5b-545aec37fd45","Type":"ContainerDied","Data":"99c543f5988ac75f9dc9ae5bd96f29713ac4b9520fa21b7f0395db33e199d626"} Oct 30 18:46:50.289879 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.289864 5070 scope.go:122] "RemoveContainer" containerID="6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf" Oct 30 18:46:50.290270 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:50.290242 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf\": container with ID starting with 6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf not found: ID does not exist" containerID="6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf" Oct 30 18:46:50.290348 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.290280 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf"} err="failed to get container status \"6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf\": rpc error: code = NotFound desc = could not find container \"6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf\": container with ID starting with 6f196956940898621fe40e8d1a3d744a2beb5841b89f1cf19eec3361bd455cbf not found: ID does not exist" Oct 30 18:46:50.290348 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.290312 5070 scope.go:122] "RemoveContainer" containerID="742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868" Oct 30 18:46:50.299288 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.299270 5070 scope.go:122] "RemoveContainer" containerID="742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868" Oct 30 18:46:50.299576 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:50.299556 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868\": container with ID starting with 742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868 not found: ID does not exist" containerID="742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868" Oct 30 18:46:50.299627 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.299582 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868"} err="failed to get container status \"742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868\": rpc error: code = NotFound desc = could not find container \"742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868\": container with ID starting with 742a70b068156c7fcf632d94aeca039ecf31d1a7640d68c8f6c5c2acbc3bf868 not found: ID does not exist" Oct 30 18:46:50.344252 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.344222 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7"] Oct 30 18:46:50.356476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.356454 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/premium-w1-a759a1-kserve-785d47b9d5-j9dp7"] Oct 30 18:46:50.393182 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.393138 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq"] Oct 30 18:46:50.396533 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.396510 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a759a1/simulator-w1-a759a1-kserve-665b7bc764-l85lq"] Oct 30 18:46:50.696102 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.696065 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:50.747064 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.747026 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4"] Oct 30 18:46:50.869524 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:50.869486 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:51.022422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:51.022327 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph"] Oct 30 18:46:51.022592 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:51.022565 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" containerID="cri-o://07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3" gracePeriod=30 Oct 30 18:46:51.034161 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:51.034124 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f01e1bb-39a4-4233-a132-6f5f2c024dac" path="/var/lib/kubelet/pods/0f01e1bb-39a4-4233-a132-6f5f2c024dac/volumes" Oct 30 18:46:51.034629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:51.034613 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f60b6bc-6a69-47e6-bb5b-545aec37fd45" path="/var/lib/kubelet/pods/7f60b6bc-6a69-47e6-bb5b-545aec37fd45/volumes" Oct 30 18:46:51.049452 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:51.049414 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:51.248558 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:51.248525 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:53.418715 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.418679 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" podUID="b5fb80ae-c927-4f2d-8212-735a691e3830" containerName="main" containerID="cri-o://61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee" gracePeriod=45 Oct 30 18:46:53.700685 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.700661 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:53.877300 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877265 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl\" (UniqueName: \"kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl\") pod \"b5fb80ae-c927-4f2d-8212-735a691e3830\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " Oct 30 18:46:53.877470 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877353 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir\") pod \"b5fb80ae-c927-4f2d-8212-735a691e3830\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " Oct 30 18:46:53.877470 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877373 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache\") pod \"b5fb80ae-c927-4f2d-8212-735a691e3830\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " Oct 30 18:46:53.877470 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877407 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs\") pod \"b5fb80ae-c927-4f2d-8212-735a691e3830\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " Oct 30 18:46:53.877470 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877460 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home\") pod \"b5fb80ae-c927-4f2d-8212-735a691e3830\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " Oct 30 18:46:53.877663 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877502 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm\") pod \"b5fb80ae-c927-4f2d-8212-735a691e3830\" (UID: \"b5fb80ae-c927-4f2d-8212-735a691e3830\") " Oct 30 18:46:53.877718 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877675 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache" pod "b5fb80ae-c927-4f2d-8212-735a691e3830" (UID: "b5fb80ae-c927-4f2d-8212-735a691e3830"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:53.877771 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877686 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir" pod "b5fb80ae-c927-4f2d-8212-735a691e3830" (UID: "b5fb80ae-c927-4f2d-8212-735a691e3830"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:53.877834 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877803 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home" pod "b5fb80ae-c927-4f2d-8212-735a691e3830" (UID: "b5fb80ae-c927-4f2d-8212-735a691e3830"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:53.877971 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877951 5070 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-tmp-dir\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:53.878030 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877975 5070 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-model-cache\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:53.878030 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.877988 5070 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-home\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:53.879821 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.879798 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm" pod "b5fb80ae-c927-4f2d-8212-735a691e3830" (UID: "b5fb80ae-c927-4f2d-8212-735a691e3830"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:53.880048 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.880022 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs" pod "b5fb80ae-c927-4f2d-8212-735a691e3830" (UID: "b5fb80ae-c927-4f2d-8212-735a691e3830"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:46:53.880187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.880028 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl" pod "b5fb80ae-c927-4f2d-8212-735a691e3830" (UID: "b5fb80ae-c927-4f2d-8212-735a691e3830"). InnerVolumeSpecName "kube-api-access-vb7jl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:53.979361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.979282 5070 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5fb80ae-c927-4f2d-8212-735a691e3830-tls-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:53.979361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.979309 5070 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5fb80ae-c927-4f2d-8212-735a691e3830-dshm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:53.979361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:53.979319 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb7jl\" (UniqueName: \"kubernetes.io/projected/b5fb80ae-c927-4f2d-8212-735a691e3830-kube-api-access-vb7jl\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:54.314827 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.314752 5070 generic.go:356] "Generic (PLEG): container finished" podID="b5fb80ae-c927-4f2d-8212-735a691e3830" containerID="61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee" exitCode=0 Oct 30 18:46:54.314827 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.314797 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" Oct 30 18:46:54.315004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.314832 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" event={"ID":"b5fb80ae-c927-4f2d-8212-735a691e3830","Type":"ContainerDied","Data":"61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee"} Oct 30 18:46:54.315004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.314870 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt" event={"ID":"b5fb80ae-c927-4f2d-8212-735a691e3830","Type":"ContainerDied","Data":"6a1e62aed5d31a8c90d4cc75e83f3290afc967bad724bffae9a0c06643040c4a"} Oct 30 18:46:54.315004 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.314886 5070 scope.go:122] "RemoveContainer" containerID="61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee" Oct 30 18:46:54.325026 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.325011 5070 scope.go:122] "RemoveContainer" containerID="61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee" Oct 30 18:46:54.325312 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:54.325292 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee\": container with ID starting with 61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee not found: ID does not exist" containerID="61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee" Oct 30 18:46:54.325359 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.325320 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee"} err="failed to get container status \"61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee\": rpc error: code = NotFound desc = could not find container \"61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee\": container with ID starting with 61c60ebbb62e35ef26faa1e379991271a3405786189fb230c07cf344879f46ee not found: ID does not exist" Oct 30 18:46:54.368623 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.368592 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt"] Oct 30 18:46:54.381704 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:54.381676 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/e2e-conflict-b-6c6935b2-kserve-86cb8d6cf5-wgznt"] Oct 30 18:46:55.032241 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:55.032211 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5fb80ae-c927-4f2d-8212-735a691e3830" path="/var/lib/kubelet/pods/b5fb80ae-c927-4f2d-8212-735a691e3830/volumes" Oct 30 18:46:55.748325 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:55.748267 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" containerName="payload-processing" containerID="cri-o://61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d" gracePeriod=25 Oct 30 18:46:55.797819 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:55.797786 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:56.288037 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.288006 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:46:56.302981 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.302916 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.302981 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.302961 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303002 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303064 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303101 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303349 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303179 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.303349 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303236 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303349 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303327 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303338 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.303503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303355 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303394 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.303503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303409 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg\") pod \"5c98abf2-a117-444a-b125-324f6c83bad7\" (UID: \"5c98abf2-a117-444a-b125-324f6c83bad7\") " Oct 30 18:46:56.303503 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303484 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:46:56.303746 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303614 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.303836 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303820 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c98abf2-a117-444a-b125-324f6c83bad7-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.303891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303843 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.303891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303857 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.303891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303870 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.303891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.303885 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.305295 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.305271 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:46:56.305424 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.305405 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.305797 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.305772 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:56.305797 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.305775 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg" pod "5c98abf2-a117-444a-b125-324f6c83bad7" (UID: "5c98abf2-a117-444a-b125-324f6c83bad7"). InnerVolumeSpecName "kube-api-access-vbhsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:56.341410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.341377 5070 generic.go:356] "Generic (PLEG): container finished" podID="5c98abf2-a117-444a-b125-324f6c83bad7" containerID="07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3" exitCode=0 Oct 30 18:46:56.341593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.341548 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" event={"ID":"5c98abf2-a117-444a-b125-324f6c83bad7","Type":"ContainerDied","Data":"07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3"} Oct 30 18:46:56.341674 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.341625 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" event={"ID":"5c98abf2-a117-444a-b125-324f6c83bad7","Type":"ContainerDied","Data":"f1c13e51f31a7b3ff99739028d87ab53100a44870d08e5c13442af0aa414ae4c"} Oct 30 18:46:56.341674 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.341658 5070 scope.go:122] "RemoveContainer" containerID="07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3" Oct 30 18:46:56.341783 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.341776 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph" Oct 30 18:46:56.356910 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.356882 5070 scope.go:122] "RemoveContainer" containerID="07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3" Oct 30 18:46:56.357279 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:46:56.357255 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3\": container with ID starting with 07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3 not found: ID does not exist" containerID="07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3" Oct 30 18:46:56.357365 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.357286 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3"} err="failed to get container status \"07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3\": rpc error: code = NotFound desc = could not find container \"07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3\": container with ID starting with 07a1cbb54034904f7727832d4798ca2d5d5ed237db3340c7488e96d261a967c3 not found: ID does not exist" Oct 30 18:46:56.372625 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.372601 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:56.405589 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.405486 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.405589 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.405512 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c98abf2-a117-444a-b125-324f6c83bad7-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.405589 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.405526 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbhsg\" (UniqueName: \"kubernetes.io/projected/5c98abf2-a117-444a-b125-324f6c83bad7-kube-api-access-vbhsg\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.405589 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.405539 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c98abf2-a117-444a-b125-324f6c83bad7-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:46:56.409924 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.409901 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph"] Oct 30 18:46:56.416032 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:56.416008 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f6f7d5-openshift-default-64d55dd877-77bph"] Oct 30 18:46:57.032499 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:57.032466 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" path="/var/lib/kubelet/pods/5c98abf2-a117-444a-b125-324f6c83bad7/volumes" Oct 30 18:46:57.492470 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:57.492435 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:57.994132 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:57.994098 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:58.066647 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:58.066613 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk"] Oct 30 18:46:58.079181 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:58.079123 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w1-a759a1-7dnwk"] Oct 30 18:46:58.153397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:58.153365 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:58.649853 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:58.649819 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:59.031682 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:46:59.031609 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7497cfa8-e00d-48cf-9cb0-2bd4825337d4" path="/var/lib/kubelet/pods/7497cfa8-e00d-48cf-9cb0-2bd4825337d4/volumes" Oct 30 18:47:00.006397 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.006351 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" podUID="70a677ad-6ac6-41a2-bb65-923f670c7048" containerName="main" containerID="cri-o://d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039" gracePeriod=45 Oct 30 18:47:00.269579 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.269524 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:47:00.347952 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.347919 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm\") pod \"70a677ad-6ac6-41a2-bb65-923f670c7048\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " Oct 30 18:47:00.348097 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.347961 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home\") pod \"70a677ad-6ac6-41a2-bb65-923f670c7048\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " Oct 30 18:47:00.348136 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348092 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb\" (UniqueName: \"kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb\") pod \"70a677ad-6ac6-41a2-bb65-923f670c7048\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " Oct 30 18:47:00.348205 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348186 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir\") pod \"70a677ad-6ac6-41a2-bb65-923f670c7048\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " Oct 30 18:47:00.348253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348232 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home" pod "70a677ad-6ac6-41a2-bb65-923f670c7048" (UID: "70a677ad-6ac6-41a2-bb65-923f670c7048"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.348253 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348244 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs\" (UniqueName: \"kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs\") pod \"70a677ad-6ac6-41a2-bb65-923f670c7048\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " Oct 30 18:47:00.348358 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348331 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache\") pod \"70a677ad-6ac6-41a2-bb65-923f670c7048\" (UID: \"70a677ad-6ac6-41a2-bb65-923f670c7048\") " Oct 30 18:47:00.348501 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348456 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir" pod "70a677ad-6ac6-41a2-bb65-923f670c7048" (UID: "70a677ad-6ac6-41a2-bb65-923f670c7048"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.348622 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348562 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache" pod "70a677ad-6ac6-41a2-bb65-923f670c7048" (UID: "70a677ad-6ac6-41a2-bb65-923f670c7048"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.348794 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348771 5070 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-tmp-dir\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.348894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348797 5070 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-model-cache\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.348894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.348811 5070 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-home\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.350309 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.350287 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm" pod "70a677ad-6ac6-41a2-bb65-923f670c7048" (UID: "70a677ad-6ac6-41a2-bb65-923f670c7048"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:00.350411 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.350376 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs" pod "70a677ad-6ac6-41a2-bb65-923f670c7048" (UID: "70a677ad-6ac6-41a2-bb65-923f670c7048"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:47:00.350411 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.350383 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb" pod "70a677ad-6ac6-41a2-bb65-923f670c7048" (UID: "70a677ad-6ac6-41a2-bb65-923f670c7048"). InnerVolumeSpecName "kube-api-access-6cgcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:00.366237 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.366214 5070 generic.go:356] "Generic (PLEG): container finished" podID="70a677ad-6ac6-41a2-bb65-923f670c7048" containerID="d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039" exitCode=0 Oct 30 18:47:00.366336 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.366264 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" Oct 30 18:47:00.366336 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.366301 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" event={"ID":"70a677ad-6ac6-41a2-bb65-923f670c7048","Type":"ContainerDied","Data":"d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039"} Oct 30 18:47:00.366442 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.366349 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2" event={"ID":"70a677ad-6ac6-41a2-bb65-923f670c7048","Type":"ContainerDied","Data":"2fa3578334b1cdeaa3fe4b337acb57de75083dbf7da28f684f7353583a0e8b76"} Oct 30 18:47:00.366442 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.366372 5070 scope.go:122] "RemoveContainer" containerID="d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039" Oct 30 18:47:00.375073 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.375058 5070 scope.go:122] "RemoveContainer" containerID="d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039" Oct 30 18:47:00.375345 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:47:00.375328 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039\": container with ID starting with d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039 not found: ID does not exist" containerID="d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039" Oct 30 18:47:00.375405 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.375359 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039"} err="failed to get container status \"d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039\": rpc error: code = NotFound desc = could not find container \"d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039\": container with ID starting with d490cfee9cc1a854f7719b5c649937f273780271e9543b8eec5bc03d04dd5039 not found: ID does not exist" Oct 30 18:47:00.420638 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.420613 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2"] Oct 30 18:47:00.426395 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.426374 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-0ddc26/simulator-w3-0ddc26-kserve-6978bb9469-wh9h2"] Oct 30 18:47:00.449906 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.449884 5070 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70a677ad-6ac6-41a2-bb65-923f670c7048-dshm\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.449906 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.449907 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cgcb\" (UniqueName: \"kubernetes.io/projected/70a677ad-6ac6-41a2-bb65-923f670c7048-kube-api-access-6cgcb\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:00.450026 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:00.449917 5070 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70a677ad-6ac6-41a2-bb65-923f670c7048-tls-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:01.065791 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:01.065757 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70a677ad-6ac6-41a2-bb65-923f670c7048" path="/var/lib/kubelet/pods/70a677ad-6ac6-41a2-bb65-923f670c7048/volumes" Oct 30 18:47:02.540629 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:02.540589 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:47:03.834087 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:47:03.834047 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:47:05.005549 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.005475 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:05.212705 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.212670 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:05.283577 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.283488 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc"] Oct 30 18:47:05.283729 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.283705 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" containerID="cri-o://cd5dff9d6d3a5492946d44d308b155b77960e4d8d4ffd13bc70550057a991288" gracePeriod=30 Oct 30 18:47:05.371667 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.371630 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:05.429628 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.429594 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv"] Oct 30 18:47:05.429861 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.429829 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" podUID="f5b89d29-0142-4973-a8bd-a39808ca2ec1" containerName="maas-api" containerID="cri-o://3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923" gracePeriod=30 Oct 30 18:47:05.574303 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.574227 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:05.732016 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.731991 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:47:05.827034 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.826954 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls\") pod \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " Oct 30 18:47:05.827191 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.827034 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca\") pod \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " Oct 30 18:47:05.827191 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.827098 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs\") pod \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " Oct 30 18:47:05.827191 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.827165 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8\" (UniqueName: \"kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8\") pod \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\" (UID: \"f5b89d29-0142-4973-a8bd-a39808ca2ec1\") " Oct 30 18:47:05.827486 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.827461 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca" pod "f5b89d29-0142-4973-a8bd-a39808ca2ec1" (UID: "f5b89d29-0142-4973-a8bd-a39808ca2ec1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:05.829109 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.829087 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8" pod "f5b89d29-0142-4973-a8bd-a39808ca2ec1" (UID: "f5b89d29-0142-4973-a8bd-a39808ca2ec1"). InnerVolumeSpecName "kube-api-access-9tvf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:05.829261 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.829239 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls" pod "f5b89d29-0142-4973-a8bd-a39808ca2ec1" (UID: "f5b89d29-0142-4973-a8bd-a39808ca2ec1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:47:05.829319 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.829255 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs" pod "f5b89d29-0142-4973-a8bd-a39808ca2ec1" (UID: "f5b89d29-0142-4973-a8bd-a39808ca2ec1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:47:05.928340 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.928300 5070 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-maas-api-tls\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:05.928340 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.928337 5070 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b89d29-0142-4973-a8bd-a39808ca2ec1-openshift-service-ca\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:05.928553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.928350 5070 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b89d29-0142-4973-a8bd-a39808ca2ec1-metrics-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:05.928553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:05.928362 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tvf8\" (UniqueName: \"kubernetes.io/projected/f5b89d29-0142-4973-a8bd-a39808ca2ec1-kube-api-access-9tvf8\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:06.035023 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.034991 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:06.191666 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.191632 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:06.438835 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.438799 5070 generic.go:356] "Generic (PLEG): container finished" podID="f5b89d29-0142-4973-a8bd-a39808ca2ec1" containerID="3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923" exitCode=0 Oct 30 18:47:06.438835 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.438841 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" Oct 30 18:47:06.439052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.438884 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" event={"ID":"f5b89d29-0142-4973-a8bd-a39808ca2ec1","Type":"ContainerDied","Data":"3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923"} Oct 30 18:47:06.439052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.438932 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv" event={"ID":"f5b89d29-0142-4973-a8bd-a39808ca2ec1","Type":"ContainerDied","Data":"7e2687dfb93e5d70fff520623f83f95d7aa6bc76362591b9bf2fee9d1f0fb1cd"} Oct 30 18:47:06.439052 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.438957 5070 scope.go:122] "RemoveContainer" containerID="3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923" Oct 30 18:47:06.449493 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.449471 5070 scope.go:122] "RemoveContainer" containerID="3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923" Oct 30 18:47:06.449748 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:47:06.449728 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923\": container with ID starting with 3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923 not found: ID does not exist" containerID="3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923" Oct 30 18:47:06.449814 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.449755 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923"} err="failed to get container status \"3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923\": rpc error: code = NotFound desc = could not find container \"3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923\": container with ID starting with 3d5260697ec6f1d79ce2aba6c0b3b1d693dcb1255f85e3cea9961548c8748923 not found: ID does not exist" Oct 30 18:47:06.500625 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.500595 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv"] Oct 30 18:47:06.513111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.513083 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-0ddc26-868c88996-7p5bv"] Oct 30 18:47:06.760964 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:06.760883 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:07.031675 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:07.031596 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5b89d29-0142-4973-a8bd-a39808ca2ec1" path="/var/lib/kubelet/pods/f5b89d29-0142-4973-a8bd-a39808ca2ec1/volumes" Oct 30 18:47:10.466075 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.466043 5070 generic.go:356] "Generic (PLEG): container finished" podID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerID="cd5dff9d6d3a5492946d44d308b155b77960e4d8d4ffd13bc70550057a991288" exitCode=0 Oct 30 18:47:10.466523 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.466084 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" event={"ID":"1161e03c-ead8-4397-bc7a-57ab809cec75","Type":"ContainerDied","Data":"cd5dff9d6d3a5492946d44d308b155b77960e4d8d4ffd13bc70550057a991288"} Oct 30 18:47:10.466523 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.466189 5070 scope.go:122] "RemoveContainer" containerID="1eb6efadf631c4dc77bfa425aa256ecb3b76b54ad58e61e49cd107a3d6fb28ab" Oct 30 18:47:10.548703 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.548677 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:47:10.676927 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.676901 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.676935 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.676969 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677022 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677104 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677159 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677199 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677228 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677284 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo\") pod \"1161e03c-ead8-4397-bc7a-57ab809cec75\" (UID: \"1161e03c-ead8-4397-bc7a-57ab809cec75\") " Oct 30 18:47:10.677350 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677282 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:10.677610 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677353 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:10.677610 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677459 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:10.677749 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677719 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.677837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677745 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.677837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677761 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.677837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.677719 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:10.678264 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.678228 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:10.679357 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.679332 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:10.679621 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.679592 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:47:10.679780 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.679757 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "kube-api-access-whqx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:10.679996 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.679974 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token" pod "1161e03c-ead8-4397-bc7a-57ab809cec75" (UID: "1161e03c-ead8-4397-bc7a-57ab809cec75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:10.778882 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.778850 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.778882 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.778878 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whqx9\" (UniqueName: \"kubernetes.io/projected/1161e03c-ead8-4397-bc7a-57ab809cec75-kube-api-access-whqx9\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.778882 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.778888 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1161e03c-ead8-4397-bc7a-57ab809cec75-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.779114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.778897 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.779114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.778905 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:10.779114 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:10.778913 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1161e03c-ead8-4397-bc7a-57ab809cec75-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:11.472767 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:11.472735 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" Oct 30 18:47:11.472767 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:11.472752 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc" event={"ID":"1161e03c-ead8-4397-bc7a-57ab809cec75","Type":"ContainerDied","Data":"30a356b02a49d0521c095872475ff39660a7f26ca2166b2724bb0cef992a45bf"} Oct 30 18:47:11.473244 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:11.472809 5070 scope.go:122] "RemoveContainer" containerID="cd5dff9d6d3a5492946d44d308b155b77960e4d8d4ffd13bc70550057a991288" Oct 30 18:47:11.544739 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:11.544704 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc"] Oct 30 18:47:11.557291 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:11.557263 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a759a1-openshift-default-6bbf9c4459-wrrmc"] Oct 30 18:47:12.540111 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:12.540077 5070 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 18:47:13.031720 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:13.031689 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" path="/var/lib/kubelet/pods/1161e03c-ead8-4397-bc7a-57ab809cec75/volumes" Oct 30 18:47:13.141529 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:13.141496 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:13.309424 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:13.309350 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:13.479431 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:13.479391 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv"] Oct 30 18:47:13.480022 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:13.479990 5070 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" containerID="cri-o://7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5" gracePeriod=30 Oct 30 18:47:13.563953 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:13.563871 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:14.878891 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:14.878862 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-trgmc_cbe4deb1-4c90-4cd7-a540-c0bcff310c70/manager/0.log" Oct 30 18:47:15.093298 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:15.093269 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-bk986_22e1e7b3-20c9-4065-b00a-ea6842e637cb/manager/0.log" Oct 30 18:47:15.558693 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:15.558646 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:16.003451 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.003426 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:47:16.130456 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.130421 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp\" (UniqueName: \"kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp\") pod \"e4a478c0-037b-4074-a788-b658798a3d8f\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " Oct 30 18:47:16.130628 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.130547 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume\") pod \"e4a478c0-037b-4074-a788-b658798a3d8f\" (UID: \"e4a478c0-037b-4074-a788-b658798a3d8f\") " Oct 30 18:47:16.130931 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.130899 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume" pod "e4a478c0-037b-4074-a788-b658798a3d8f" (UID: "e4a478c0-037b-4074-a788-b658798a3d8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:16.132605 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.132579 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp" pod "e4a478c0-037b-4074-a788-b658798a3d8f" (UID: "e4a478c0-037b-4074-a788-b658798a3d8f"). InnerVolumeSpecName "kube-api-access-drdbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:16.231463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.231424 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drdbp\" (UniqueName: \"kubernetes.io/projected/e4a478c0-037b-4074-a788-b658798a3d8f-kube-api-access-drdbp\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:16.231463 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.231458 5070 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4a478c0-037b-4074-a788-b658798a3d8f-plugins-config-volume\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:16.505028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.504939 5070 generic.go:356] "Generic (PLEG): container finished" podID="e4a478c0-037b-4074-a788-b658798a3d8f" containerID="61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d" exitCode=0 Oct 30 18:47:16.505028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.504993 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" Oct 30 18:47:16.505028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.505017 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" event={"ID":"e4a478c0-037b-4074-a788-b658798a3d8f","Type":"ContainerDied","Data":"61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d"} Oct 30 18:47:16.505302 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.505094 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4" event={"ID":"e4a478c0-037b-4074-a788-b658798a3d8f","Type":"ContainerDied","Data":"37a842910fa4f754e957df2b8e6359b16b4349faf9f6676e1e162d9072e01a2e"} Oct 30 18:47:16.505302 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.505118 5070 scope.go:122] "RemoveContainer" containerID="61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d" Oct 30 18:47:16.514719 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.514702 5070 scope.go:122] "RemoveContainer" containerID="61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d" Oct 30 18:47:16.514967 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:47:16.514946 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d\": container with ID starting with 61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d not found: ID does not exist" containerID="61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d" Oct 30 18:47:16.515013 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.514974 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d"} err="failed to get container status \"61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d\": rpc error: code = NotFound desc = could not find container \"61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d\": container with ID starting with 61a618702264793ef6c72aa4176d28aaab258862cf2aaef0465927a7f5faf44d not found: ID does not exist" Oct 30 18:47:16.562428 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.562404 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4"] Oct 30 18:47:16.574474 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:16.574454 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-0ddc26-ccfb77f7d-ndmf4"] Oct 30 18:47:17.031450 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:17.031416 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" path="/var/lib/kubelet/pods/e4a478c0-037b-4074-a788-b658798a3d8f/volumes" Oct 30 18:47:17.535783 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:17.535751 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-xwz9c_568c6bf1-8dd1-4aeb-a18d-cebf58d57a56/manager/0.log" Oct 30 18:47:17.859823 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:17.859720 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:18.049246 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.049217 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:18.682074 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.682036 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:18.739806 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.739784 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:47:18.856461 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856383 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856461 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856460 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856500 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856547 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856583 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856598 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856639 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.856690 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856667 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.857009 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856695 5070 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert\") pod \"622b1121-a027-4243-be20-75eb8da3137b\" (UID: \"622b1121-a027-4243-be20-75eb8da3137b\") " Oct 30 18:47:18.857009 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.856974 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.857113 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.857062 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.857239 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.857197 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.857239 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.857212 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.857434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.857298 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:47:18.858634 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.858613 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:47:18.858829 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.858801 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:18.859062 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.859034 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "kube-api-access-45wgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:47:18.859169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.859065 5070 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo" pod "622b1121-a027-4243-be20-75eb8da3137b" (UID: "622b1121-a027-4243-be20-75eb8da3137b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:47:18.958176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958105 5070 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958139 5070 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-credential-socket\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958176 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958183 5070 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/622b1121-a027-4243-be20-75eb8da3137b-istiod-ca-cert\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958194 5070 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-envoy\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958202 5070 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/622b1121-a027-4243-be20-75eb8da3137b-istio-podinfo\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958211 5070 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45wgl\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-kube-api-access-45wgl\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958220 5070 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-istio-data\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958229 5070 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/622b1121-a027-4243-be20-75eb8da3137b-workload-certs\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:18.958410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:18.958236 5070 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622b1121-a027-4243-be20-75eb8da3137b-istio-token\") on node \"ip-10-0-130-102.ec2.internal\" DevicePath \"\"" Oct 30 18:47:19.017721 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.017678 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:19.523059 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.523025 5070 generic.go:356] "Generic (PLEG): container finished" podID="622b1121-a027-4243-be20-75eb8da3137b" containerID="7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5" exitCode=0 Oct 30 18:47:19.523553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.523074 5070 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" Oct 30 18:47:19.523553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.523105 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" event={"ID":"622b1121-a027-4243-be20-75eb8da3137b","Type":"ContainerDied","Data":"7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5"} Oct 30 18:47:19.523553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.523171 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv" event={"ID":"622b1121-a027-4243-be20-75eb8da3137b","Type":"ContainerDied","Data":"a58e24dfbf528a85eaba8bd25c1aae4685c696e69e6a4181eac7f85bb06d2e1e"} Oct 30 18:47:19.523553 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.523195 5070 scope.go:122] "RemoveContainer" containerID="7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5" Oct 30 18:47:19.532384 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.532368 5070 scope.go:122] "RemoveContainer" containerID="7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5" Oct 30 18:47:19.532619 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:47:19.532598 5070 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5\": container with ID starting with 7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5 not found: ID does not exist" containerID="7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5" Oct 30 18:47:19.532662 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.532631 5070 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5"} err="failed to get container status \"7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5\": rpc error: code = NotFound desc = could not find container \"7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5\": container with ID starting with 7c6e6b1461d9764f827430da43b3085a491fdf54615652c81f35223df57b26e5 not found: ID does not exist" Oct 30 18:47:19.596205 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.596176 5070 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv"] Oct 30 18:47:19.609010 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:19.608978 5070 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-0ddc26-openshift-default-7f95765489-ww7zv"] Oct 30 18:47:21.031376 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:21.031344 5070 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="622b1121-a027-4243-be20-75eb8da3137b" path="/var/lib/kubelet/pods/622b1121-a027-4243-be20-75eb8da3137b/volumes" Oct 30 18:47:21.161275 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:21.161238 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:22.314915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:22.314877 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:22.487841 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:22.487809 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:23.479187 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:23.479098 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:24.163062 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:24.163025 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:25.173547 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:25.173488 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:29.407649 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:29.407617 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:33.832810 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:47:33.832773 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:47:36.404497 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:36.404459 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-5022a55d-lbfp9_ff6409e0-4b88-4ebc-8ef2-22dad5bfc027/revoke-keys/0.log" Oct 30 18:47:37.801467 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:37.801437 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-77bbf9c4-2knjh_4c0be8c5-bdf6-4844-8bf7-65fa97f4d5ae/revoke-keys/0.log" Oct 30 18:47:38.585895 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:38.585857 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8f0806aa-s2r8h_15911411-0b80-46ef-ac4b-1ff42527fcfb/revoke-keys/0.log" Oct 30 18:47:39.372542 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:39.372513 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-xgfxv_a0adcbf7-e3ee-47f4-8b85-b092efc0df10/postgres/0.log" Oct 30 18:47:41.264694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:41.264661 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-trgmc_cbe4deb1-4c90-4cd7-a540-c0bcff310c70/manager/0.log" Oct 30 18:47:41.484323 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:41.484292 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-bk986_22e1e7b3-20c9-4065-b00a-ea6842e637cb/manager/0.log" Oct 30 18:47:43.138434 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:43.138393 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-xwz9c_568c6bf1-8dd1-4aeb-a18d-cebf58d57a56/manager/0.log" Oct 30 18:47:43.971697 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:43.971660 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:47:45.087954 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:45.087922 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:47:45.290617 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:45.290586 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:47:45.904744 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:45.904713 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:47:46.531444 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:46.531413 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:47:46.939769 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:46.939725 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:47:47.538510 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:47.538479 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:47:48.803140 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:48.803108 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-cns7p_f915fff9-985c-4353-b5fb-166e126440cb/main/0.log" Oct 30 18:47:49.011650 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:49.011621 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-jffqg_58701ee9-4eb9-4637-a40e-b1611586455d/main/0.log" Oct 30 18:47:49.778786 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:49.778756 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-xdsnd_837eb667-3457-409a-9818-d502ec96f06e/main/0.log" Oct 30 18:47:49.972180 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:47:49.972130 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-5fpf4_c1c41d57-c5a2-419c-8e72-67ae1cc874ea/main/0.log" Oct 30 18:48:00.449098 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.449031 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:00.565207 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565169 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tq9m/must-gather-f2r4t"] Oct 30 18:48:00.565592 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565580 5070 cpu_manager.go:414] "containerMap: removing container" podUID="0f01e1bb-39a4-4233-a132-6f5f2c024dac" containerName="main" Oct 30 18:48:00.565636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565593 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f01e1bb-39a4-4233-a132-6f5f2c024dac" containerName="main" Oct 30 18:48:00.565636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565603 5070 cpu_manager.go:414] "containerMap: removing container" podUID="b5fb80ae-c927-4f2d-8212-735a691e3830" containerName="main" Oct 30 18:48:00.565636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565608 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5fb80ae-c927-4f2d-8212-735a691e3830" containerName="main" Oct 30 18:48:00.565636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565619 5070 cpu_manager.go:414] "containerMap: removing container" podUID="70a677ad-6ac6-41a2-bb65-923f670c7048" containerName="main" Oct 30 18:48:00.565636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565625 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70a677ad-6ac6-41a2-bb65-923f670c7048" containerName="main" Oct 30 18:48:00.565636 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565634 5070 cpu_manager.go:414] "containerMap: removing container" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565639 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565649 5070 cpu_manager.go:414] "containerMap: removing container" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" containerName="payload-processing" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565654 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" containerName="payload-processing" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565660 5070 cpu_manager.go:414] "containerMap: removing container" podUID="f5b89d29-0142-4973-a8bd-a39808ca2ec1" containerName="maas-api" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565664 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5b89d29-0142-4973-a8bd-a39808ca2ec1" containerName="maas-api" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565678 5070 cpu_manager.go:414] "containerMap: removing container" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565682 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565687 5070 cpu_manager.go:414] "containerMap: removing container" podUID="7f60b6bc-6a69-47e6-bb5b-545aec37fd45" containerName="main" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565692 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f60b6bc-6a69-47e6-bb5b-545aec37fd45" containerName="main" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565697 5070 cpu_manager.go:414] "containerMap: removing container" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565702 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565713 5070 cpu_manager.go:414] "containerMap: removing container" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565717 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565726 5070 cpu_manager.go:414] "containerMap: removing container" podUID="15911411-0b80-46ef-ac4b-1ff42527fcfb" containerName="revoke-keys" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565731 5070 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15911411-0b80-46ef-ac4b-1ff42527fcfb" containerName="revoke-keys" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565801 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f01e1bb-39a4-4233-a132-6f5f2c024dac" containerName="main" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565809 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5fb80ae-c927-4f2d-8212-735a691e3830" containerName="main" Oct 30 18:48:00.565812 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565816 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="70a677ad-6ac6-41a2-bb65-923f670c7048" containerName="main" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565821 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="622b1121-a027-4243-be20-75eb8da3137b" containerName="istio-proxy" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565827 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4a478c0-037b-4074-a788-b658798a3d8f" containerName="payload-processing" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565832 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5b89d29-0142-4973-a8bd-a39808ca2ec1" containerName="maas-api" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565838 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565843 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f60b6bc-6a69-47e6-bb5b-545aec37fd45" containerName="main" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565847 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c98abf2-a117-444a-b125-324f6c83bad7" containerName="istio-proxy" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565853 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="15911411-0b80-46ef-ac4b-1ff42527fcfb" containerName="revoke-keys" Oct 30 18:48:00.566332 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.565986 5070 memory_manager.go:365] "RemoveStaleState removing state" podUID="1161e03c-ead8-4397-bc7a-57ab809cec75" containerName="istio-proxy" Oct 30 18:48:00.569206 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.569189 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.590020 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.589997 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tq9m\"/\"openshift-service-ca.crt\"" Oct 30 18:48:00.597496 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.597465 5070 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6tq9m\"/\"kube-root-ca.crt\"" Oct 30 18:48:00.597733 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.597708 5070 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6tq9m\"/\"default-dockercfg-vlv4w\"" Oct 30 18:48:00.599847 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.599822 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/must-gather-f2r4t"] Oct 30 18:48:00.620319 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.620291 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:00.672475 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.672439 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7aef3d00-5b43-4986-a412-ef10e4e74882-must-gather-output\") pod \"must-gather-f2r4t\" (UID: \"7aef3d00-5b43-4986-a412-ef10e4e74882\") " pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.672646 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.672494 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w62wt\" (UniqueName: \"kubernetes.io/projected/7aef3d00-5b43-4986-a412-ef10e4e74882-kube-api-access-w62wt\") pod \"must-gather-f2r4t\" (UID: \"7aef3d00-5b43-4986-a412-ef10e4e74882\") " pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.773637 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.773552 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w62wt\" (UniqueName: \"kubernetes.io/projected/7aef3d00-5b43-4986-a412-ef10e4e74882-kube-api-access-w62wt\") pod \"must-gather-f2r4t\" (UID: \"7aef3d00-5b43-4986-a412-ef10e4e74882\") " pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.773786 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.773711 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7aef3d00-5b43-4986-a412-ef10e4e74882-must-gather-output\") pod \"must-gather-f2r4t\" (UID: \"7aef3d00-5b43-4986-a412-ef10e4e74882\") " pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.774059 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.774041 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7aef3d00-5b43-4986-a412-ef10e4e74882-must-gather-output\") pod \"must-gather-f2r4t\" (UID: \"7aef3d00-5b43-4986-a412-ef10e4e74882\") " pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.798046 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.798017 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w62wt\" (UniqueName: \"kubernetes.io/projected/7aef3d00-5b43-4986-a412-ef10e4e74882-kube-api-access-w62wt\") pod \"must-gather-f2r4t\" (UID: \"7aef3d00-5b43-4986-a412-ef10e4e74882\") " pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:00.882907 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:00.882870 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/must-gather-f2r4t" Oct 30 18:48:01.207883 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:01.207855 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/must-gather-f2r4t"] Oct 30 18:48:01.209035 ip-10-0-130-102 kubenswrapper[5070]: W1030 18:48:01.209010 5070 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7aef3d00_5b43_4986_a412_ef10e4e74882.slice/crio-33050594ca1ab6a6d5543fbcffec2cdd05ddb0d966e32602511a8aa71eeec961 WatchSource:0}: Error finding container 33050594ca1ab6a6d5543fbcffec2cdd05ddb0d966e32602511a8aa71eeec961: Status 404 returned error can't find the container with id 33050594ca1ab6a6d5543fbcffec2cdd05ddb0d966e32602511a8aa71eeec961 Oct 30 18:48:01.211301 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:01.211284 5070 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:48:01.750817 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:01.750773 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/must-gather-f2r4t" event={"ID":"7aef3d00-5b43-4986-a412-ef10e4e74882","Type":"ContainerStarted","Data":"33050594ca1ab6a6d5543fbcffec2cdd05ddb0d966e32602511a8aa71eeec961"} Oct 30 18:48:02.767447 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:02.767408 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/must-gather-f2r4t" event={"ID":"7aef3d00-5b43-4986-a412-ef10e4e74882","Type":"ContainerStarted","Data":"4d6d1f3bc1a678be4399bd01e1d791cf97eacfcd79a3b8791a276195b3bf85d7"} Oct 30 18:48:02.767856 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:02.767461 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/must-gather-f2r4t" event={"ID":"7aef3d00-5b43-4986-a412-ef10e4e74882","Type":"ContainerStarted","Data":"8ba5261d881db361ad89aa38dd5a95bc4a6322d54718bec80c7ab997468f31c8"} Oct 30 18:48:02.810723 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:02.810670 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-6tq9m/must-gather-f2r4t" podStartSLOduration=2.037844355 podStartE2EDuration="2.81065645s" totalImagesPullingTime="772.812095ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:02.808368295 +0000 UTC m=+999.923502215" watchObservedRunningTime="2026-10-30 18:48:02.81065645 +0000 UTC m=+999.925790762" Oct 30 18:48:03.838589 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:48:03.838554 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:48:05.084994 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:05.084962 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jw7nd_deaa4bf3-e4f9-4b22-9bda-eba0080c0697/global-pull-secret-syncer/0.log" Oct 30 18:48:05.534287 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:05.534259 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-9q5g7_0fae4e48-2851-4c67-9615-ad86f9795397/connection-checker/0.log" Oct 30 18:48:05.650151 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:05.650110 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-twcqb_9c695b41-2900-4b0d-832b-bb35456eb816/connection-checker/0.log" Oct 30 18:48:05.764926 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:05.764898 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-xzmd9_a04618ce-f5a4-49ae-8fee-ffb082167b8c/connection-checker/0.log" Oct 30 18:48:05.893476 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:05.893446 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" Oct 30 18:48:06.259293 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:06.259216 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-102.ec2.internal_bdb7686e846fff2d163fb7282f45a982/haproxy/0.log" Oct 30 18:48:09.429254 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:09.429189 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:09.591903 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:09.591869 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:11.184053 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:11.183989 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:14.325294 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:14.325261 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:14.577220 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:14.573452 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:15.069639 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.069605 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:15.135706 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.135673 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:15.279178 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.278997 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:15.316644 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.316613 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:15.360672 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.360582 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:15.512188 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.512161 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:15.542063 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.542031 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:15.638058 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.638013 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:15.737094 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.737004 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:15.852361 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.852333 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:15.867225 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:15.867192 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:16.045544 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:16.045418 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:18.480017 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:18.479943 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-trgmc_cbe4deb1-4c90-4cd7-a540-c0bcff310c70/manager/0.log" Oct 30 18:48:18.550116 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:18.550088 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-bk986_22e1e7b3-20c9-4065-b00a-ea6842e637cb/manager/0.log" Oct 30 18:48:18.631593 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:18.631545 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-trgmc_cbe4deb1-4c90-4cd7-a540-c0bcff310c70/manager/0.log" Oct 30 18:48:18.656377 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:18.656347 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-trgmc_cbe4deb1-4c90-4cd7-a540-c0bcff310c70/manager/0.log" Oct 30 18:48:18.715165 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:18.714566 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-bk986_22e1e7b3-20c9-4065-b00a-ea6842e637cb/manager/0.log" Oct 30 18:48:18.747579 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:18.747492 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-bk986_22e1e7b3-20c9-4065-b00a-ea6842e637cb/manager/0.log" Oct 30 18:48:19.166694 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:19.166659 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:19.196410 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:19.196377 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-xwz9c_568c6bf1-8dd1-4aeb-a18d-cebf58d57a56/manager/0.log" Oct 30 18:48:19.572271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:19.572199 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-xwz9c_568c6bf1-8dd1-4aeb-a18d-cebf58d57a56/manager/0.log" Oct 30 18:48:19.763732 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:19.763688 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-xwz9c_568c6bf1-8dd1-4aeb-a18d-cebf58d57a56/manager/0.log" Oct 30 18:48:20.107119 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.107079 5070 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc"] Oct 30 18:48:20.115423 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.115390 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.134422 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.132015 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc"] Oct 30 18:48:20.170914 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.170867 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-podres\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.170914 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.170917 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-lib-modules\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.171211 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.170958 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-proc\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.171211 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.170988 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-sys\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.171211 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.171095 5070 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4pw5\" (UniqueName: \"kubernetes.io/projected/f65177dd-4d0a-4de8-860a-8e1c8f26b198-kube-api-access-q4pw5\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272567 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-proc\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272606 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272613 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-sys\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272697 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-proc\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272714 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4pw5\" (UniqueName: \"kubernetes.io/projected/f65177dd-4d0a-4de8-860a-8e1c8f26b198-kube-api-access-q4pw5\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272773 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-sys\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272806 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-podres\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272835 5070 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-lib-modules\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.272916 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272860 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-podres\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.273123 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.272949 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f65177dd-4d0a-4de8-860a-8e1c8f26b198-lib-modules\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.299740 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.299705 5070 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4pw5\" (UniqueName: \"kubernetes.io/projected/f65177dd-4d0a-4de8-860a-8e1c8f26b198-kube-api-access-q4pw5\") pod \"perf-node-gather-daemonset-97stc\" (UID: \"f65177dd-4d0a-4de8-860a-8e1c8f26b198\") " pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.442450 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.442330 5070 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:20.768083 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.767545 5070 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc"] Oct 30 18:48:20.923884 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:20.923845 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" event={"ID":"f65177dd-4d0a-4de8-860a-8e1c8f26b198","Type":"ContainerStarted","Data":"785f7bd061ea635fff6622e4587e33e9775be4a9ba1ff1e52f2947519143b4fe"} Oct 30 18:48:21.605525 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:21.605444 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:21.799782 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:21.799743 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-n922x_06297482-f53e-497f-8012-1c6a55b1462b/cluster-monitoring-operator/0.log" Oct 30 18:48:21.932952 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:21.932913 5070 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" event={"ID":"f65177dd-4d0a-4de8-860a-8e1c8f26b198","Type":"ContainerStarted","Data":"923a16a94c3b3051ce287949a88c94576682a7effe41a7a23911b2f15ec2dca0"} Oct 30 18:48:21.933138 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:21.933055 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:21.976711 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:21.976655 5070 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" podStartSLOduration=1.976641479 podStartE2EDuration="1.976641479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:21.974018244 +0000 UTC m=+1019.089152162" watchObservedRunningTime="2026-10-30 18:48:21.976641479 +0000 UTC m=+1019.091775395" Oct 30 18:48:22.777753 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:22.777431 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg6zs_7990fc23-358e-4104-b308-12adc1dcbc9a/node-exporter/0.log" Oct 30 18:48:22.880750 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:22.880724 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg6zs_7990fc23-358e-4104-b308-12adc1dcbc9a/kube-rbac-proxy/0.log" Oct 30 18:48:22.985028 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:22.984999 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hg6zs_7990fc23-358e-4104-b308-12adc1dcbc9a/init-textfile/0.log" Oct 30 18:48:23.221915 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:23.221884 5070 scope.go:122] "RemoveContainer" containerID="bc0344b25360be56704a85c4a79f71dcd9a3b56a946c9e30db911f2370621f92" Oct 30 18:48:24.800369 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:24.800339 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-pprwl_2969e9ce-6c93-4660-a5ee-050390b1aec8/prometheus-operator-admission-webhook/0.log" Oct 30 18:48:25.644931 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:25.644904 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-597b95cb84-nkx9x_1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9/cert-manager-controller/0.log" Oct 30 18:48:25.709433 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:25.709382 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-cainjector-5c57bbd4fc-wq5qd_ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1/cert-manager-cainjector/0.log" Oct 30 18:48:25.958346 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:25.958254 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-597b95cb84-nkx9x_1e6bdef6-7869-49a0-ac7d-8fb9b50ab5b9/cert-manager-controller/0.log" Oct 30 18:48:26.013683 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:26.013650 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/cert-manager_cert-manager-cainjector-5c57bbd4fc-wq5qd_ac8ada95-cc5f-42fe-8bbe-0e3224de5ae1/cert-manager-cainjector/0.log" Oct 30 18:48:26.370926 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:26.370899 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:26.787212 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:26.787122 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-gb874_267afe49-5c51-4731-b7a6-d2d14070d443/manager/0.log" Oct 30 18:48:26.827305 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:26.827280 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-gb874_267afe49-5c51-4731-b7a6-d2d14070d443/copy-manifests/0.log" Oct 30 18:48:27.002169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:27.001515 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-gb874_267afe49-5c51-4731-b7a6-d2d14070d443/manager/0.log" Oct 30 18:48:27.043894 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:27.043812 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-6cbccd5bdc-gb874_267afe49-5c51-4731-b7a6-d2d14070d443/copy-manifests/0.log" Oct 30 18:48:27.334377 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:27.334288 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-glvc2_2fdf2c77-42d8-4686-a656-334557d55531/networking-console-plugin/0.log" Oct 30 18:48:27.950905 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:27.950879 5070 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6tq9m/perf-node-gather-daemonset-97stc" Oct 30 18:48:28.245169 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:28.245079 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/1.log" Oct 30 18:48:28.248729 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:28.248711 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-c262b_eb14d800-4d5f-4a7c-bade-81c0f4f5c3f3/console-operator/2.log" Oct 30 18:48:29.245170 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:29.245120 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-xsmnj_7a1e5faa-e3b4-4764-bd0d-6717de32fda3/download-server/0.log" Oct 30 18:48:29.625403 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:29.625351 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:29.764231 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:29.764201 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:30.325962 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.325933 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-c4nbz_9f0e7740-17a0-460e-9407-5650fa708046/volume-data-source-validator/0.log" Oct 30 18:48:30.425779 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.425749 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:30.486737 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.486706 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:30.526271 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.526238 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:30.584348 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.584271 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:30.667120 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.667093 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:30.763038 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.763007 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:30.850827 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.850749 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:30.942039 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.942008 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:30.980960 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:30.980932 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:31.077024 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:31.076997 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:31.163425 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:31.163376 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:31.246815 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:31.246737 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:32.343002 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:32.342978 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-585f46ff54-gstjh_fa302d0e-a330-40b7-bd14-6bd262531f62/manager/0.log" Oct 30 18:48:32.481492 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:32.481462 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-585f46ff54-gstjh_fa302d0e-a330-40b7-bd14-6bd262531f62/manager/0.log" Oct 30 18:48:32.618383 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:32.618351 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lpq29_79d5b339-6665-4fbe-92db-77fd202119d2/dns/0.log" Oct 30 18:48:32.729333 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:32.729304 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lpq29_79d5b339-6665-4fbe-92db-77fd202119d2/kube-rbac-proxy/0.log" Oct 30 18:48:33.270546 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:33.270522 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sm8gj_463c6228-727e-4d4f-9b80-1db615657b4f/dns-node-resolver/0.log" Oct 30 18:48:33.841059 ip-10-0-130-102 kubenswrapper[5070]: E1030 18:48:33.841024 5070 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-srvzm" containerName="extractor" Oct 30 18:48:34.357581 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:34.357549 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-566c94599c-nvz29_a5550d6f-177d-40fe-8a26-7a369d3b5208/registry/0.log" Oct 30 18:48:34.459130 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:34.459101 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9nr4w_11d10c3e-223f-41ef-ac3f-cd428bf43ebd/node-ca/0.log" Oct 30 18:48:36.390425 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:36.390308 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:37.160658 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:37.160633 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:37.283627 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:37.283598 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:37.393506 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:37.393474 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:37.715106 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:37.715081 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:37.823986 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:37.823957 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:37.887439 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:37.887414 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:38.050976 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.050901 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:38.067587 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.067556 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:38.257234 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.257206 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:38.280619 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.280596 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:38.382571 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.382543 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:38.558946 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.558914 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:38.593282 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:38.593253 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:40.055736 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.055699 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-trgmc_cbe4deb1-4c90-4cd7-a540-c0bcff310c70/manager/0.log" Oct 30 18:48:40.110541 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.110514 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-bk986_22e1e7b3-20c9-4065-b00a-ea6842e637cb/manager/0.log" Oct 30 18:48:40.227837 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.227806 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr9ld_68a3d167-dbc7-4155-aee3-e056b558ab8a/serve-healthcheck-canary/0.log" Oct 30 18:48:40.493798 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.493759 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-xwz9c_568c6bf1-8dd1-4aeb-a18d-cebf58d57a56/manager/0.log" Oct 30 18:48:40.623113 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.623081 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7chc8s4_979f3204-6db1-4a7d-9863-7b11c566154e/istio-proxy/0.log" Oct 30 18:48:40.759078 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.758990 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:40.922404 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:40.922358 5070 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-plcdm_8b4ec318-a75a-4dd0-9da0-ffc41310b641/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:41.223559 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.223517 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-zkkkh_13093b2f-a782-4521-b80f-f4ee194af653/insights-operator/0.log" Oct 30 18:48:41.223958 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.223938 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-zkkkh_13093b2f-a782-4521-b80f-f4ee194af653/insights-operator/1.log" Oct 30 18:48:41.298259 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.298226 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-6njrd_72ebd198-f18a-4a53-a8d8-8ac880573147/istio-proxy/0.log" Oct 30 18:48:41.302746 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.302727 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srvzm_cbf77bfc-9322-4635-9080-36631d7613df/kube-rbac-proxy/0.log" Oct 30 18:48:41.362252 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.362224 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-cvhp5_7e87a8d3-748e-4674-9da3-e54a3ce24b87/istio-proxy/0.log" Oct 30 18:48:41.405246 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.405218 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srvzm_cbf77bfc-9322-4635-9080-36631d7613df/exporter/0.log" Oct 30 18:48:41.509477 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.509379 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-srvzm_cbf77bfc-9322-4635-9080-36631d7613df/extractor/0.log" Oct 30 18:48:41.555258 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.555230 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-qqbk5_f32b9d16-e1d8-4d54-9434-261254241d83/istio-proxy/0.log" Oct 30 18:48:41.747722 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.747688 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-tzpc7_7d625d9d-56f7-4149-b9e2-5a8b5f114aaf/istio-proxy/0.log" Oct 30 18:48:41.880153 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:41.880127 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-h92rd_92a34c34-38fc-4dc4-a72b-79a371564b82/istio-proxy/0.log" Oct 30 18:48:42.069204 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:42.069175 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75fdfd5b99-vpjd7_1adf30d9-26c7-46b0-b9bb-dfb96ba3711e/router/0.log" Oct 30 18:48:42.259106 ip-10-0-130-102 kubenswrapper[5070]: I1030 18:48:42.259005 5070 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-6jh2k-wg5tb_89a201b9-118c-43aa-8be1-1329c22d9130/insights-gathering/0.log"